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ABSTRACT 


The methodology evaluation and recommendations presented in this 
document are part of an overall effort initiated by the Army to improve its 
Hardw are versus Man power (HARDMAN) methodology for projecting manpower, 
personnel, and training (MPT) to support new acquisitions. Several different 
validity tests were employed to evaluate the methodology. These tests 
included (1) a general examination of the basic reasonableness of the 
technique; (2) a reliability measure of how well other trained individuals, 
given the same data inputs, can derive the same answers as an already- 
completed test case; (3) a qualitative accuracy check based on talking to 
individuals who have already applied the same, or similar, methodology; and 
(4) a simple measure of how accurately the methodology must predict the real 
world. The net outcome of the study revealed that the methodology conformed 
fairly well with both the Army's MPT user needs and other accepted manpower 
modeling techniques. Additionally, audits of three completed HARDMAN 
applications revealed only a small number of potential problem areas compared 
to the total number of issues investigated. The reliability study results 
conformed well with the problem areas uncovered through the audits. The major 
problems discovered basically revolved around (1) a need to tighten up select 
judgmental portions of the methodology, and (2) a requirement to improve those 
aspects of the technique used for making MPT support and budget decisions. 

The results of the accuracy studies suggested, respectively, that although no 
firm accuracy judgment could be made at this time, the manpower life-cycle 
cost component was only marginally sensitive to changes in other related cost 
variables. Therefore, even with some minor problems, the methodology seemed 
to be sound and to have good near-term utility to the Army. Recommendations 
were provided to firm up the problem areas revealed through the evaluation. 
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FOREWORD 


This document provides the results of the first phase of an evaluation 
effort designed to examine and improve the Army's HARDMAN manpower projection 
methodology. The work was performed by JPL and was sponsored by the Army 
Research Institute for the Behavioral and Social Sciences (ARI) through a 
reimbursable agreement (ARI Nos. 13ARI19 Change 1 and 13ARI83-35, respectively 
dated February 15 and March 9, 1983) with the National Aeronautics and Space 
Administration (Task RD-182, Amendment 256). Dr. John Whittenburg, Team 
Leader for the Systems Manning Technical Area, is the task officer. 

In a literal sense, this document is the product of a team effort. 

Wayne Zimmerman, the Task Manager and prime author, was assisted by several 
colleagues. Valerie Gray was responsible for the user requirements portion of 
the study and assisted in performing the reliability study. Leigh Rosenberg 
organized and conducted the reliability study. As an adjunct to the 
reliability study, Robert Butler, of the Assessment Group consulting firm, 
performed the event-series validity evaluation. Mark Franklin and Steven 
Parks participated and provided assistance, respectively, in the SINCGARS 
replication and technical audit efforts, which were part of the validity 
studies. Shirley Stroup was responsible for typing the document. 
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SECTION I 


INTRODUCTION AND SUMMARY 


A. OBJECTIVES 

The Jet Propulsion Laboratory (JPL), sponsored by the Army Research 
Institute (ARI), was asked to provide the Army with an objective, independent 
assessment of the validity and reliability of the HARDMAN ( Hardw are vs. 
Man power) Comparability Methodology (HCM) for projecting manpower, personnel, 
and training. JPL was also asked to make recommendations for needed improve- 
ments in the methodology that would ultimately help the Army implement an 
accurate manpower, personnel, and training (MPT) system that could be applied 
early in the acquisition process of new systems. 

The following paragraphs discuss the major assumptions and constraints 
that established the framework of the study, the background behind the Army 
manpower and training issue, the evaluation approach and summary of results, 
and the overall structure of the document. 


B. BACKGROUND 

Having stated the overall objectives of the study, it is appropriate at 
this time to provide a brief summary of the background of the task. 

Over the past several years, the Army has recognized a critical manpower 
situation revolving around the widening gap between increasing weapon com- 
plexity and declining manpower availability. The problem of declining 
manpower availability has been further aggravated by the apparent decay of 
entrant skill levels. An effort was initiated by ARI in 1980, as part of an 
Army-wide interest to improve the manpower problem, to determine whether a 
Navy methodology called HARDMAN (HARDware vs. MANpower) could be applied to 
Army systems. The Army was interested in determining whether an existing 
manpower, personnel, and training projection technique could be modified to 
provide early, accurate inputs to the acquisition process. The Army chose to 
examine the HCM (one of several methodologies developed under the auspices of 
the Navy HARDMAN Office) because it was an integrated MPT technique and had 
already been used in several Navy acquisition programs. The other Navy 
methodologies appeared to be too narrow in scope or required input data not 
available under the Army's existing data collection systems. Although these 
systems have not been ruled out for Army use, it seemed that for the near 
term, the greatest payoff rested on the HCM. The HCM contains the following 
six major steps: 

(1) Establish a consolidated data base composed of functional descrip- 
tions of the proposed and similar predecessor systems; and 
associated inputs such as hardware reliability, personnel informa- 
tion, costs, and training data. This information is used to 
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support succeeding steps in the analysis. This step is essential 
because the Army does not usually organize and store data in 
effective structure to conduct human resource analysis. 

. (2) Determine the manpower requirements and skills necessary to 
operate and maintain the system. 

(3) Determine training requirements likely to be imposed by the 
proposed system. 

(4) Determine personnel requirements (i.e., personnel flows that will 
be necessary to support the manpower requirements). 

(5) Conduct an impact analysis to establish likely manpower and 
training shortages and identify parts of the system that represent 
high-cost drivers. 

(6) Perform a tradeoff analysis to alter features of the system (e.g., 
reliability or required skill levels) to reduce or eliminate 
unreasonable demands. 

Each of these steps involves several judgments pertaining to the 
selection of comparable predecessor systems, the identification and assembly 
of associated data on the predecessor systems, expected performance of the 
proposed system as compared to predecessor systems, and required skills. 
Additionally, gaps in the supporting data often exist at the conceptual design 
stage. At these junctions, expert judgment is exercised to merge the various 
pieces of information into a comprehensive, consistent picture to be used to 
generate the personnel projections. The Army's concern is that these judgment 
areas could act as major error sources in the manpower projection. 
Consequently, JPL was asked to perform an objective evaluation of the 
methodology to identify both strengths and weaknesses of the technique and 
make recommendations, as appropriate. 


C. ASSUMPTIONS AND CONSTRAINTS 

One of the methodology validity tests involved a comparison of the user 
needs against the proposed outcomes of the projection scheme. To establish 
the user requirements, a detailed interview process was designed to use the 
bank of experience already existing in the Army. The major assumption 
involved with the user requirements effort centered on the user contacts. 
Because the user contacts in the various Army commands were established via 
the sponsor and recommendations from the users themselves, it was assumed that 
the suggested sources were sufficiently experienced and knowledgeable in the 
MPT arena so that their answers were representative of the respective commands 
in general. Even though the sample size was small, the consistency of the 
responses suggested that the answers were a good representation of the Army as 
a whole. 
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Several limitations were imposed on the study. The size of the sample 
user population was limited primarily by the available time to complete the 
evaluation. Some of the users contacted were more informed or aware than 
others about MPT processes or projection models such as the HCM. Therefore, 
it was understood that some of their responses could reflect an already 
established preference, depending on their knowledge of the various elements 
present in the HCM or other MPT models. To circumvent this bias, the 
questionnaire and interview technique employed in this evaluation focused on 
getting the user to provide requirements for a "generic" MPT projection system 
for the Army. 

It should be noted that the user requirements determined by this study 
reflect the needs of a present or near-future MPT system. Any changes to this 
environment could result in additional or different requirements (e.g., 
implementation of Integrated System Support (ISS)). Therefore, it is 
important that the Army consider periodic reevaluation of its MPT requirements 
to ensure that any in-place MPT system remains responsive to the needs of the 
various MPT players. 

Although it was originally planned to do a thorough validity and 
reliability evaluation of the methodology, internal time and budget limita- 
tions resulted in the sponsor opting to defer (1) the accuracy check on the 
MPT methodology, (2) the life-cycle cost (LCC) and methodology sensitivity 
analysis, and (3) the quantitative accuracy check on the methodology input 
data (e.g., reliability and maintainability data). Finally, other duty 
commitments required that the internal validity experiment be simplified so 
that the Army participants could finish the replication within a maximum 
three-week time frame. 


D. SUMMARY OF EVALUATION APPROACH AND RESULTS OF ANALYSIS 

The preceding discussion outlined the various steps and judgments 
necessary to exercise the HCM. Because of the breadth and complexity of MPT 
issues covered by the technique, a multifaceted evaluation was structured to 
examine potential validity aspects. The validity aspects, evaluation 
approach, and study results are summarized in the following paragraphs. 


1. Evaluation Approach 

As stated earlier in this section, time and budget constraints 
limited the original scope of the study. Validity measures related to testing 
model accuracy were deferred to the next study phase. However, within the 
imposed constraints several other validity tests were conducted. Face 
validity, a measure of whether the model appears logical and reasonable, was 
the first aspect investigated. Although not a rigorous credibility test, a 
face-validity study does afford some insights into the general utility of the 
model, potential error sources, data availability problems, logic foundation, 
and possible ambiguities in logic structure or results. The approach for 
performing the face-validity examination was fourfold. First, a user 
requirements study was performed to establish the Army's needs within the MPT 
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community. A total of nine distinct user groups (thirteen individuals) were 
interviewed in person to establish user MPT needs in areas, ranging from the 
basic design of the MPT projection system to the kinds of information desired 
from the system outputs. A basic utility test was then performed by comparing 
each user requirement against the same respective element in the HCM. Second, 
existing DOD and Army directives were examined to see whether there were any 
established, accepted MPT techniques presently in use that could be used to 
confirm the general logic structure and data foundation. Third, a larger 
literature search was conducted, encompassing other government agencies and 
the private sector. Again, the object was to develop a firm basis for 
evaluating the logic and data support structures of the HCM, as well as 
examining potential sources for error. Last, detailed audits were conducted 
on three applications of the methodology. Independent analysts attempted to 
replicate major empirical and judgmental steps in each application with the 
intent of examining overall logic, potential sources of errors, data 
manipulation or availability problems, and ambiguities in how the projections 
were derived and interpreted. 

Another test, internal reliability, was employed to get an understanding 
of the ability of others to successfully apply the technique and obtain 
consistent answers. An internal validity test is essentially a measure of the 
variance in model outputs when a group of independent analysts replicate an 
already completed application given the same inputs. Ultimately, the 
objective is to establish the clarity and succinctness of the model, as well 
as the ability to have any reasonably trained individual perform the process 
reliably. An actual Army system was selected for the internal validity test. 

A group of seven individuals (four civilians and three Army officers) were 
initially trained in the execution of the HCM and then provided with data 
packages that allowed them to conduct a '’limited" manpower and personnel 
assessment on the proposed system (i.e., because of other duty commitments and 
resultant time restrictions, only four major components of the system were 
studied). 

It was decided that an attempt should be made to examine model accuracy 
from at least a qualitative standpoint. To this end, several individuals in 
both the Air Force and Navy, who had experience with applications of the same 
(or similar) MPT model, were contacted for information pertaining to any tests 
or actual comparative field data that might indicate the relative accuracy of 
the methodology. 

Finally, the last validity aspect examined was an event-series test. 
Event-series validity is basically a measure of how accurately a simulation 
must reproduce a real-world event. A top-level, life-cycle, cost-sensitivity 
analysis, keying on the manpower variable, was employed as the tool to achieve 
this measure. The required accuracy of the manpower variable was determined 
by performing iterations of the life-cycle cost projection (using an example 
acquisition) until all other dependent variables, manning, and total 
life-cycle cost fell within prescribed error bounds. This measure was 
particularly important for obtaining a further understanding of the model 
usefulness as a decision making tool. 
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Throughout all of these validity exercises several strengths and parallel 
problem areas surfaced. The complete results of the evaluation are summarized 
in the following discussion. 


2. Summary of Results 

Table 1-1 provides an overall summary of the study results. Each 
of the above evaluation areas is discussed briefly in the following paragraphs. 


Table 1-1. Net Results of HARDMAN Evaluation 


Evaluation Areas 

Net Results 

User Requirements 

Technique complied with majority of mandatory 
user requirements 

Operational Analysis 
Comparison with 
other MPT methods 

Methodology conformed with other known, accepted 
MPT modeling schemes, and data foundations 

Audits 

Methodology demonstrated sound logic and 
reasonable results for 85% of test issues 
examined (remaining issues considered reparable 
in near term) 

Reliability Analysis 
Internal reliability 

Test group correctly replicated half of selected 
test points (two of the four remaining test 
points found to be repeatable after clarification) 

Qualitative 
Accuracy Check 

Two of nine individuals having experience with 
HARDMAN applications indicated a rough accuracy 
of 80-90% with actual manning requirements; 
remaining individuals had insufficient experience 
to comment 

Event Series Validity 

Manpower variable found to be least sensitive of 
all life-cycle cost variables, demonstrating good 
utility of existing methodology even without 
suggested near-term improvements 
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a. User Requirements Analysis . The user requirements analysis 
generated a total of twenty-two mandatory MPT system requirements. Thirteen 
of the mandatory requirements applied to system outputs, and the remaining 
nine requirements were related to system design. An additional four require- 
ments were given lower priority by the users and were therefore listed as 
"desirable." The major thrust of the user needs revolved around the key design 
drivers of affordability, supportability , and sustainability. By comparison, 
it was found that the HCM complied with seventeen of the twenty-two mandatory 
requirements. The areas of noncompliance were as follows: 

(1) Not indicating the accuracy of the MPT projection system to aid 
budget control and program-level decisions . 

(2) Not including demographic data such as age, sex, size, etc., as 
part of the man-machine matching and support process. 

(3) Not considering the demands of other acquisitions on the available 
manpower pool (i.e., the HCM still focuses primarily on the 
individual equipment system). 

(4) Not performing a complete life-cycle cost tradeoff analysis (i.e., 
the HCM focuses only on MPT costs). 

(5) Not indicating weaknesses in the MPT analysis along with 
respective resolution actions. 

Overall, it seems that the HCM is reasonably responsive to the Army's 

needs . 


b. Operational Analysis . The literature search and audit 
aspects of the face validity study were jointly conducted under the 
operational analysis portion of the evaluation. This portion of the overall 
study was broadly termed "operational analysis" because the ensuing 
investigation required a rigorous exercising of the HCM. The review and 
comparison of related DOD and Army directives provided three major findings: 
(1) DOD system acquisition guidelines stated a requirement to consider MPT in 
the design process but did not appear to afford a means of projecting MPT 
requirements; (2) the Army Manpower Authorization, Criteria (MACRIT) guide did 
provide a simple manpower projection algorithm; and (3) the HCM appeared to be 
designed around the MACRIT manpower algorithm, with its outputs generally 
organized to conform with broader DOD acquisition directives. These insights, 
although serving as good initial data points, only addressed a fraction of the 
total methodology. The follow-up government and private-sector literature 
searches were far more intensive. Nineteen other MPT models and papers, 
encompassing all aspects of the HCM, were studied. Generally, it was 
discovered that the other modeling approaches were reasonably equivalent in 
their assumptions, algorithms, and data inputs. In one area, the personnel 
pipeline projection, the HCM approach was found to be stronger because it 
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employed a modeling technique that reduced the need for dif f icult-to-obtain 
data such as present and future manpower availabilities. On the negative side, 
it seemed that there were four distinct areas where potential inaccuracies 
could be interjected: (1) The use of the MACRIT manpower algorithm could 
result in early or late life-cycle manpower projection errors unless system 
deployment and phase-out rates are considered; (2) non-consideration of major 
socio-economic factors such as the draft, economy, change in federal govern- 
ment administrations, or massive military reorganizations could influence the 
attrition and retention characteristics of personnel career paths; (3) the 
non-use of some type of long-term manpower availability projection to indicate 
shortages at the time of system deployment could result in sizeable personnel 
errors; and (4) the lack of a complete life-cycle cost model could prevent the 
full range of potential cost and system design tradeoffs from being provided 
to the decision makers. 

The last aspect of the face-validity analysis, the audits, revealed that 
the model successfully passed eighty-five percent of the test issues examined. 
The remaining potential problem areas discovered were (1) the lack of a 
structured process of selecting, adjusting, and manipulating the source data 
could sizeably affect the accuracy and credibility of the projections; (2) the 
use of a single reference system (composed of similar predecessor components) 
to model several widely varying proposed designs could also affect the 
accuracy and credibility of the results; (3) the absence of an adjustment to 
the complete predecessor historical data to consider the design improvements 
of the proposed system means that it would not be feasible to make a valid 
comparison between real world performance and the often overly-optimistic 
projection of the contractor; (4) the lack of a structured process for 
approximating the value of peculiar design changes or induced human error as 
related to component failure rates presents a means of interjecting computa- 
tional mistakes as well as reducing both the audit and usability capacities of 
the method; (5) the absence of indicators for data quality and projection 
accuracy could affect the credibility and usefulness of the results; and (6) 
the complexity of the technique requires the use of a well-seasoned, multi- 
disciplinary team which, in turn, could impact the ability of others to apply 
the method. 


c. Reliability Analysis . The last parts of evaluation, the 
internal and event-series validity tests and qualitative accuracy check, were 
performed under the reliabiity analysis. This was done because all of these 
validity aspects related to both the internal and external reliability of the 
process. The results of the internal validity check were closely aligned with 
the audit findings. As a whole, the test group discovered that, given the 
same inputs, it could not replicate the original SINCGARS results. Although 
all of the study group seemed to understand roughly half of the early steps in 
the process reasonably well, they had problems with (1) developing a reference 
system that jointly met the functional requirements and was representative of 
the two widely varying contractor designs; (2) not understanding the thought 
process for determining impacts of design differences, and how and when to 
perturb historical predecessor data to model components of the proposed 
system; and (3) not comprehending the exact process of matching or selecting 
skills. After items (1) and (3) were clarified, the study group had a fairly 
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good grasp of the majority of the early steps in the process. The test 
subjects uniformly agreed that none of them singularly had enough training or 
educational background to exercise the complete methodology with any 
confidence . 

The major finding in the event-series validity study was that the MPT 
projections could easily vary by plus or minus fifteen percent without causing 
major problems at the upper DOD budget and cost control management levels. 
Component usage rate was found to have the greatest influence on causing 
unacceptable errors in manpower projections. Although it was jointly dis- 
covered that there is still not sufficient historical experience to make a 
firm qualitative accuracy judgment on the process, the low sensitivity finding 
on the manpower variable was a promising indicator of the utility of the model 
even with its minor drawbacks. 


d. Conclusions and Recommendations . In conclusion, it was felt 
that, overall, the HCM appeared to be reasonably sound. The flaws discovered 
did not appear to be irreparable. The major near-term recommendations 
suggested to tighten up the problem areas were as follows: (1) Establish 
subjective criteria for rating the quality of various data sources; (2) 
provide an indication of the credibility of the projections; (3) establish a 
top-level method for comparing the technology of the reference and proposed 
systems on an equal basis; (4) establish solid guidelines and information 
sources for building reference systems and consider the use of two reference 
systems to model two widely varying contractor-proposed designs; (5) firm up 
guidelines for the functional requirements level of indenture; (6) provide a 
better structure for MOS and skill selection; (7) periodically update manpower 
projections (every two to four years) over a system's life cycle to offset 
possible end-point errors; (8) provide a more structured tradeoff process; and 
(9) standardize the assessment of study weaknesses and resolution actions. 

The intermediate term (two years from the present) priorities were 

(1) project manpower availability out to the actual system deployment date; 

(2) incorporate a more complete life cycle cost tradeoff analysis; (3) firm up 
ranges for perturbation values, induced failures, and indirect productivity 
factors . 

Far-term recommendations (for the three- to five-year time frame 
following the present) included (1) consideration of other emerging system 
manpower demands; (2) development of a quantitative means to differentiate 
between critical and non-critical tasks; (3) expansion of the demographic data 
base; and (4) consideration of major socioeconomic fluctuations. Potential 
solutions for all of the preceding recommendations were also provided. 

In addition to these recommendations, it was also suggested that, based 
on the audit and reliability findings, future HARDMAN training programs solicit 
a multidisciplinary team of individuals with backgrounds in the military, 
engineering, information systems and data management, and personnel. Further, 
combinations of two of more of these disciplines in each individual as part of 
the overall multidisciplinary team seem to form the best foundation for accu- 
rately exercising the methodology. 
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E. STRUCTURE OF DOCUMENT 

This document is structured so that a brief history of both the HARDMAN 
effort and Army involvement in HARDMAN is provided first in Section II as a 
basis for the study. Section III discusses the overall design of the analysis 
and broadly explains the purpose and content of the various validity studies. 
Sections IV through VI provide the detailed results of the methodology evalua- 
tion in the major areas of user requirements, face validity (operational 
analysis), and reliability (internal and event-series validity). Finally, 
Section VII discusses the implications of the results and provides the overall 
assessment of the technique with recommendations. 
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SECTION II 


HARDMAN BACKGROUND 


A. OVERVIEW 

The previous section provided a brief summary of the evaluation approach 
and results. As stated earlier in Section I, the Army has been concerned 
about the widening gap between growing technology and decreasing availability 
of needed skills. Manpower guidelines already exist through DOD documents 
such as the 5000 series and Military Standard (MIL-STD) 1388. These documents, 
which establish acquisition and support procedures for new systems, address 
the requirement to have a proper man-machine balance but do not specify 
techniques to accomplish the balance. Given the ever-widening technology and 
skills gap, the Army has been pressed to develop clear, accurate manpower 
projection techniques to fill the void in existing DOD acquisition guidelines. 
The following paragraphs provide a brief summary of the history and progression 
of events leading up to the Army's decision to develop an appropriate MPT pro- 
jection system. 


B. HARDMAN HISTORY 

The weapon system acquisition process has evolved over a long period of 
time. The overall system acquisition strategy suggests that the number of 
competing alternatives should be narrowed by eliminating those concepts no 
longer considered viable as a function of four major measures (1,2): (1) the 

ability of a design to meet the proposed mission; (2) the compatibility of the 
design with the anticipated threat; (3) a comparison of each concept with 
operational requirements; and (4) a comparison of each design's threshold 
values (i.e., cost, performance, readiness, and supportability impacts). 

These measures are employed throughout the system development process. The 
development process consists of four phases containing three major milestones 
or acquisition decision points (2): 

(1) Phase I (Milestone I) - Concept development. 

(2) Phase II (Milestone II) - Concept demonstration and 

validation. 

(3) Phase III (Milestone III)- System development. 

(4) Phase IV - System production 

and deployment. 

At the end of each phase or milestone, a go/no-go decision is made by 
the Army System Acquisition Review Council (ASARC) and Defense System Acquisi- 
tion Review Council (DSARC) as to whether to proceed to the next milestone 
(1,2). A key consideration in all of the preceding measures and subsequent 
decision points is the integration of manpower, personnel, and training (MPT) 
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into new designs. Recent studies (3, 4) by some of the services (i.e., Air 
Force and Navy) indicate that operations and support costs can consume from 40 
to 50% of the total life-cycle cost of a weapon system. Additionally, the 
manpower component of the support portion of life-cycle cost can consume as 
much as 60%. These studies underline the growing importance of MPT. 

Although the Air Force was the first service to develop a major thrust 
to resolve the MPT dilemma, the actual formalization of the HARDMAN concept, 
as presently defined, started with the creation of Navy's HARDMAN Development 
Office in 1977 (5). The Navy's HARDMAN effort was initiated partially because 
of studies similar to References 3 and 4 and to the fact that present DOD 
directives do not adequately define methods for projecting MPT requirements 
for new acquisitions. The first Navy HARDMAN study, initiated in 1975, was 
funded by the Chief of Naval Operations (CNO) Studies and Analysis Program. 
This study established that (1) systems were being designed without adequate 
attention given to MPT costs and (2) logistics support planning (and in 
particular MPT) for new acquisitions was inadequate. The CNO study outlined 
the following solutions: 

(1) Develop analytical tools and methods for MPT to influence the 
weapon system acquisition process. 

(2) Develop an MPT information system. 

(3) Revise existing directives governing the acquisition process to be 
more sensitive to MPT. 

As a result of the CNO study and some aggressive management, the Navy 
established the HARDMAN office and set about developing a modeling system 
composed of four linked components consistent with the aforementioned CNO 
guidelines and tailored to the various design phases. 

The linked modeling system, which is the present HARDMAN design, 
basically consisted of the following: 

(1) A manpower projection model. 

(2) A training resource assessment. 

(3) A life-cycle cost analysis. 

(4) A HARDMAN information system. 

The manpower projection link of the system (HCM) was designed by 
Dynamics Research Corporation (DRC) and was developed around the concept of 
using information from similar predecessor technology as a basis for manpower 
projections (6). Drawing on an extensive historical data base as well as the 
new design inputs, the manpower for the proposed system is estimated by 
altering the historical data as a function of the peculiar design improvements 
and fitting these results into a simple manpower equation consisting 
principally of total workload manhours and total systems to be procured. The 
HCM proceeds to establish required skills, training, MP costs, and potential 
personnel shortages through the remaining steps in the methodology. MPT 
estimates are then refined as the input data matures through subsequent design 
stages . 
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The PACER Corporation developed the training resource link (HTRRDM), 
designed to provide a checklist and straightforward algorithms for defining 
elements such as training sites, instructors, instructional devices, course 
material, and training costs (7). 

The life-cycle cost link of the HARDMAN system (HLCCM) was developed by 
the Assessment Group firm and designed in two levels of detail to be com- 
mensurate with the different stages of design development. Drawing on standard 
life-cycle cost algorithms, a skill-cost model, and designer inputs, the tech- 
nique provides both the total-system, life-cycle cost and the ability to 
quickly perform life-cycle cost tradeoffs between various logistic elements (8). 

The last system link, the HARDMAN Information System (HIS), was designed 
by Ackman Associates. Although not completed, the HIS is structured to be the 
repository for all predecessor system MPT and life-cycle cost data as well as 
data for new potential acquisitions. 

The Navy effort has considerable value for several reasons. Most 
important is the clear demonstration of the necessity to bond MPT securely 
with the acquisition community. Another clear value of the Navy's effort 
revolves around the many manpower models developed by DOD and the various 
services over the past several years. These models generally have only 
addressed select aspects of the MPT problem such as manpower flows in and out 
of various skill areas or total manpower requirements. The Navy's thrust has 
been to develop a total MPT system, which extends beyond these smaller 
studies. Additionally, the Navy's program has produced a large bank of 
experience and MPT tools. 


C. ARMY INVOLVEMENT IN HARDMAN 

Like the Navy, the Army has been observing similar problems with the 
mismatch between advancing technology and decreasing availability of skills. 

In 1980, the Army Research Institute (ARI) initiated an effort, in support of 
the Army-wide interest in the system manning problem, to establish whether the 
Navy HARDMAN effort could be tailored for use on new Army acquisitions. 
Recognizing that approximately seventy five percent of the system acquisition 
cost is committed by Phase II of the development cycle, the Army was also 
interested in determining whether the technique could provide early, 
reasonably accurate MPT estimates. As indicated in the preceding section, 
there was considerable overlap between the comparability, training, and 
costing links. Because it appeared that the HCM was the most encompassing and 
mature of the MPT models, ARI decided to perform some test applications of 
this technique as well as a validation of the methodology. To date, the Army 
has funded four test applications: Corps Support Weapon System (CSWS), 

Division Support Weapon System (DSWS), Remotely Piloted Vehicle (RPV), and 
Single Channel Ground Airborne Radio System (SINCGARS). The study performed 
in this document represents the first segment of the Army's validation effort 
and draws on the results of three of the test applications. The HCM presently 
represents a possible near-term solution to the Army's MPT dilemma. 


2-3 



D. DESCRIPTION AND INTENDED USE OF THE ARMY HARDMAN SYSTEM 

The Army HARDMAN system is presently emphasizing only the HCM. The 
technique consists of six major steps as shown in Figure 2-1 (6). Step 1, 
Establish Consolidated Data Base, includes the development of functional 
descriptions of the alternate systems and the collection of the input data 
(hardware, personnel, training, and cost) needed to support the succeeding 
analyses. Step 2, Determine Manpower Requirements, refers to the process of 
identifying the number and skills of soldiers necessary to operate and 
maintain the system. Step 3, Determine Training Requirements, is done in 
parallel with Step 4 and requires the specification of the numbers and types 
of instructors, courses, media, and training devices, etc., which are 
necessary to accommodate the new equipment. Step 4, Determine Personnel 
Requirements, addresses the activity of identifying the peculiar occupational 
speciality requirements. Step 5, Conduct Impact Analysis, focuses on the 
activity of comparing assessed demands (from Steps 2-4) and available supplies 
and notes unreasonable resource demands in terms of both manpower and training 
costs and personnel. Step 6, Tradeoff Analysis, refers to the activity of 
altering features of the system and repeating Steps 1 through 5 to reduce or 
eliminate unreasonable demands. 

Within each step there are several judgments required that are basically 
satisfied by using experience from the new system designers or a group of 
outside experts (6). For example, in Step 1, the evaluator(s) is required to 
interpret documents and use personal judgment in establishing the functional 
requirements for the new system and in selecting similar predecessor 
components that most closely meet these requirements. In Step 2, generic 
kinds of tasks must be defined and assigned to the various echelons of 
maintenance (i.e., organizational level in the field and intermediate level 
removed from the field). Step 2 also requires the evaluator(s) to make 
educated guesses as to the split between inherent versus induced component 
failures, design differences, selection and assignment of skills for the 
various tasks, and probable system and component usage rates. In Step 3, the 
analyzer must decide which kinds of predecessor training resources are 
applicable to the proposed system and, where no training scenario exists, 
generate new training resource requirements. Steps 4 and 5 involve judgments 
about the movement and availability of people both within various skills and 
the Array as a whole and also require the development of options to offset high 
demands. Step 6 does not require any major judgments other than a selection 
of different operating, reliability, or manning scenarios that represent a 
modified system designed to offset high manpower demands or costs. 

In reviewing the Army MPT problem outlined earlier in this section, it 
appears that a technique such as the HCM could certainly contribute to 
resolving most of the man-machine mismatches with new acquisitions. 

Ultimately, it is the Army's primary intent to use the results of such an 
approach to influence the ASARC, DSARC, and system program management 
communities to be more supportive of the MPT portion of system-integrated 
logistics support. At a secondary level, it is also the Army's desire to 
examine techniques such as the HCM with the intent of determining the 
feasibility of enhancing and managing its own MPT system. For both the ASARC 
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Figure 2-1. Steps in Application of the HARDMAN Methodology 
















decision and internal MPT management objectives it is important that the Army 
establish a credible MPT projection scheme. It then follows that any 
anticipated MPT projection techniques be validated before being approved for 
Army-wide use. The following section describes the structure for the HCM 
validity evaluation and sets the foundation for the final results provided 
later in the report. 
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SECTION III 


STRUCTURE OF HARDMAN ANALYSIS 


A. OVERVIEW 

The previous section described how important it is for the Army to 
develop a credible, comprehensive MPT system. There are several different 
kinds of validity tests to consider when establishing the true credibility of 
a modeling technique. For example, some kinds of mathematical models may obey 
other already proven algorithmic or empirical relationships. Validating 
techniques using this method is rather simple. However, the problem becomes 
more difficult when a proposed simulation is projecting the future based on 
experiential judgment and scant, or non-existent, historical evidence. As 
shown in Section II, the proposed Army MPT methodology not only fits this 
description but is further complicated by the need to draw on a considerable 
amount of data and data sources. In situations such as this there are several 
different kinds of validity tests that can be employed to test "first-time" 
models. In this section the various kinds of validity tests are first 
described, followed by a detailed discussion of the structure of each portion 
of the evaluation in the context of the validity tests. 


B. VALIDITY TESTING 

The two important outcomes of testing the validity of a simulation or 
projection are the degree of credibility of the model and results, and the 
respective usefulness of the results. The credibility of a projection 
methodology revolves around several operations research-type measures. Two 
very important validity measures are as follows: 

(1) Variable parameter validity - a measure of how well the simulation 
parameters compare with their respective counterparts in the real, 
or observable, world; and whether the parameters, when varied over 
an expected range, effect the results in the direction of known 
historical data. 

(2) Hypothesis validity - a measure of how well various relationships 
within the model and its outcomes correspond with the real world. 

Both of these validity measures fit under generic tests of "model 
accuracy." As stated earlier in Section I-B, the detailed accuracy 
examination has been postponed to a later phase. 

Within the constraints of this study, the following validity tests were 
exercised: 

(1) Face validity - a general indication of whether the model is 

logical, credible, and the results reasonably based on the inputs. 
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This indication can be obtained by talking to individuals 
experienced with the same, or similar, models; comparing the 
structure of the model to other established simulations; or 
tracing and evaluating test case results for reasonableness. 

(2) Internal reliability - a measure of the variance in model outputs 
when replicated by a group of independent assessors given the same 
inputs . 

(3) Event series validity - It is understood that inherent within the 
design of any model is the inability to duplicate exactly the real 
world. Event series validity is a measure of how accurately the 
simulation must reproduce real-world events. This test is 
extremely important because it relates to both the credibility and 
utility of the results as viewed by the people using the outputs 
for their decision process (i.e., the ASARC and Program Managers). 

In the following paragraphs, the various analytic procedures employed in 
evaluating the Army MPT methodology are discussed in terms of the preceding 
validity tests. 


C, USER REQUIREMENTS ANALYSIS 

One of the more obvious tests of face validity is a straightforward, 
initial comparison of the user needs against the outputs of the model. By 
querying knowledgeable individuals in the Army MPT arena for problems, and 
their respective needs to resolve those problems, it was possible to evaluate 
whether the proposed methodology could illuminate the key personnel issues. 

The effort designed to establish the relevance of the HCM MPT system to the 
Army's needs was fourfold: 

(1) Establish a list of experienced, interested individuals from the 
Army MPT community and attempt to draw on individuals from as many 
participating commands as possible to establish a broad user base. 

(2) Design a comprehensive set of questions to probe both major MPT 
problem areas and needed solutions and/or tools to resolve 
personnel issues. 

(3) Obtain answers to these questions via personal interviews with 
each individual. 

(4) Compare the results against the applicable portions of the 
methodology to establish whether HARDMAN is reasonably responsive 
to the stated issues. 

Each step in the above user analysis, along with both the user require- 
ments and relevance results, is discussed in greater detail in Section IV. 
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D. 


OPERATIONAL ANALYSIS 


As a continuation of the face-validity evaluation, it is also useful to 
examine applications of the methodology as a means of establishing credibility. 
In this study the examination encompasses both actual test cases and comparison 
with other accepted or proven MPT projection techniques. The structure and 
results of both of the aforementioned examinations are provided in greater 
detail later in Section V. However, as a basis for the outcomes provided in 
Section V, it is important to note that the results afforded insight into 
several credibility issues. First, the results provided information on how 
rational the model was. For example, the test-case audits were established to 
examine the consistency and reasonableness of the results relative to the 
inputs. Similarly, by comparing the technique against other accepted models, 
it was possible to see whether the basic assumptions and logic structure 
appeared sound. Second, the audit results provided useful information on the 
complexity, and subsequent utility, of the model. Third, the audits provided 
an understanding of whether the model logic and interpretation of the 
projections were perhaps ambiguous or masked, due to model over-complexity. 
Fourth, the audits indicated the presence and relative occurrence of error 
sources in the methodology. Last, the use of both test cases and comparisons 
with other MPT models highlighted potential problems in input data quality and 
availability. 


E. RELIABILITY ANALYSIS 

An examination of the "reliability" or accuracy of the methodology and 
its projections was the original intent of this portion of the evaluation. 
Although it was possible to retain part of the original reliability analysis 
scope, time and funding constraints required the actual accuracy effort to be 
postponed until the next evaluation phase. The portions retained for this 
part of the evaluation address the issues of internal reliability, a pre- 
cursory accuracy examination using the experience of individuals who have 
already applied the HCM, or a similar methodology, and event-series validity. 

For the internal reliability test, it was decided to use one of the Army 
test cases (i.e., SINCGARS) and design a controlled experiment to establish 
whether a group of individuals, initially trained to use the methodology, 
could both replicate the original answers and demonstrate a low variance in 
answers among themselves. A group of seven individuals (four civilians and 
three career Army personnel) of varying backgrounds were selected for the 
experiment. Given the small sample size (primarily due to restrictions in 
available people, time, and budget) it was decided to dispense with detailed 
statistical comparisons and simply establish general conclusions about the 
repeatability and consistency of the results. The replication was also used 
to develop a list of critical factors that have major impacts on the MPT 
projections . 

To round out the accuracy examination within the constraints of the 
study, it was also decided to perform a qualitative accuracy check, using past 
Air Force and Navy experience with the same type of methodology. 
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As stated earlier, the event-series validity test is extremely important 
for determining the required accuracy of a model and, subsequently, the 
utility of a new simulation. To obtain an estimate for the required accuracy 
of the HCM, a simple top-down cost sensitivity analysis was employed to 
establish the allowable variance in the manpower projection, as constrained by 
other life-cycle cost elements and an allowable error band around the total 
life-cycle cost projection. This estimate of the allowable variance in the 
manpower projection was performed for an example acquisition, using actual 
Army weapon system data (i.e., the SINCGARS system). Once the level of 
accuracy was established, it was possible to formulate corrective actions (for 
the various problem areas revealed through the face-validity and internal 
reliability investigations) commensurate with a greater or lesser need for 
accuracy. It was for this reason that the sensitivity test was reserved for 
the end of the evaluation. The complete details of the internal validity and 
accuracy studies are provided in Section VI. 
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SECTION IV 


USER REQUIREMENTS ANALYSIS 


A. OVERVIEW 

As a part of the approach developed to determine the face validity of 
the HCM MPT projection model, an MPT user requirements study was conducted. 
This analysis included a definition and documentation of Army user require- 
ments for a manpower, personnel, and training (MPT) prediction model. Sub- 
sequently, the user requirements were compared with the capabilities of the 
HCM to determine the degree to which the technique conformed with the needs of 
intermediate as well as end users of MPT information. The users included 
those who currently develop the MPT information as well as those who primarily 
use the information (Table 4-1). The details of the requirements analysis are 
provided in the following paragraphs. 


B. APPROACH 

This study focused on exploring Army user requirements for a "generic" 
MPT methodology that would serve as comparison against an existing MPT pro- 
jection technique for the Army's Integrated Logistic Support (ILS) process. 
The user requirements provided a useful measure for determining the extent to 
which the HARDMAN model fulfilled the Army's stated requirements. As stated 
earlier in Section III-D and III-E, detailed audits and a test case replica- 
tion were also conducted to assess face validity and internal reliability. 

The additional familiarization with the HCM technique achieved through these 
parallel analyses augmented the comparison of the user requirements with the 
methodology. 

The basic approach, outlined in the preceding discussion, is explained 
in greater detail in the following paragraphs: 


1. Selection of Individuals 

The Interim Report for Manpower and Personnel Requirements 
Determination Methodologies (MANPERS) states that "the three principal 
organizations responsible for the research, development, and deployment policy 
of a system under development are DARCOM - the materiel developer; TRADOC - 
the training and doctrine (i.e., combat) developer; and HQDA - the force 
modernization planner. With the exception of the "industry" user, all of the 
users interviewed in this study were from these three organizations in the 
Army. These commands were also confirmed by the sponsoring agencies (Army 
Research Institute (ARI) and the Soldier Support Center (SSC)), as key actors 
in the Array's MPT process. 

A predominance of users were from the TRADOC community. This is a 
reflection of the fact that "front end analysis," such as MPT projections, 
done prior to Phase I in the systems acquisition process, is largely a TRADOC 
responsibility. TRADOC has the lead in ILS management prior to the appoint- 
ment of a DARCOM Project Manager at Milestone I. 
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Table 4-1. User Groups/Representatives Interviewed 


Users 


Number of 
Interviews 


ILS Managers (DARCOM) 

Soldier Support Center/SSC-NCR (TRADOC) 

Logistics Center/LOGCEN (TRADOC) 

Deputy Chief of Staff, Combat Development /DCSCD (TRADOC) 
Deputy Chief of Staff, Training/DCST (TRADOC) 

Special Study Group-TRADOC Systems MGR/SSG-TSM (TRADOC) 

Logistics Evaluation Agency-DCSLOG/LEA 

DCSPER 

Industry (based on results of survey) 


2 

3 

1 

1 

2 

1 

1 

1 

1 


Total User Groups: 9 


Total Respondents 3 : 13 


a Users listed were selected on the basis of their role/function in the MPT 
process. All users contacted participated. 


Inputs from the SSC-NCR-II (TRADOC) and industry users were not gathered 
from actual interviews . The information needed from these users was not 
obtainable within the allocated time frame for the study and was, therefore, 
extracted from documents that adequately portrayed their respective user needs 
for an MPT projection model. One document was a memorandum about requirements 
for an MPT model (9) and the other was an industry survey conducted by the 
Soldier Support Center to determine industry requirements for MPT information 
in developing proposals (10). 


2. Questionnaire Design 

The questionnaire was designed as a discussion guide to identify 
user requirements. Some questions addressed user requirements directly by 
identifying the specific MPT information input they require to fulfill their 
responsibilities for the Army MPT process. Other requirements directly 
addressed were imbedded in questions concerning the user's interpretations of 
the timing and interfaces required by the process. Through interface and 
timing questions, an understanding was developed about when critical MPT 
information is needed and about the relationships between principal organiza- 
tional players, so that it could be determined if the design or structuring of 
an MPT projection system would be sensitive to these relationships. Another 
direct way of focusing on user requirements was to ask respondents for 
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suggestions about the design of an MPT methodology. Suggestions included 
comments on outputs, schedule, and the process of actually projecting per- 
sonnel needs. Questions indirectly addressing user needs provided confirma- 
tion, context, or further elaboration of the direct user data. These questions 
referred to the following contextual information: 

(1) The user's understanding of the present technique used for 
MPT projections. 

(2) Users seen as responsible for MPT projections and the nature 
of their "use" of the information (intermediate or end use). 

(3) MPT regulations /documents that impact the user. 

(4) Users' perceptions of the strengths or weaknesses of the 
current MPT process. 

(5) The primary charter and responsibilities of the user. 

(6) The key drivers underlying the need for MPT projections, 
according to each user interviewed. 

The complete questionnaire is shown in Appendix A. 


3. Interview Procedure 

User requirements were determined primarily through interviews 
with the representative users who were identified and recommended by the 
sponsor. Initial contacts made by ARI provided JPL with direct access to 
appropriate individuals with whom formal follow-up requests for assistance 
were made and visits scheduled. 

The study received a substantial level of user cooperation in conducting 
the interviews; an example was that users frequently called upon people within 
their organizations having specialized knowledge of that organization's 
involvement in the MPT process. Each scheduled interview was treated as a 
single-user interview even if more than one person was present for the 
discussion. A single-interview form consolidated the responses of all present 
from that organization. 


C. TABULATION OF INTERVIEWS 
1. General Responses 

Most of the users interviewed thought that MPT projections should 
be available at the front end of the acquisition cycle. This was frequently 
linked to supportability and affordability drivers for needing MPT information, 
particularly because these affect the user organization's roles and responsi- 
bilities for MPT projections. One user cited sustainability as a key driver 
of MPT information. This was considered an important insight because it 
highlights life-cycle aspects of system operation. The predominant responses 
pertaining to these three major drivers are summarized in Table 4-2. 
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Table 4- 

■2. Key MPT Drivers and Associated User Rationale 

Key MPT Drivers 

Predominant Rationale 

Affordability 

MPT system must provide all MPT cost elements associated 
with acquisitions 

MPT system must reflect cost tradeoffs with force 

_ 

structure/mission requirements 

Supportability 

MPT system must support readiness goals 

MPT system must fully support new acquisitions 
(operations, maintenance, supply, training, etc.) 

MPT system must demonstrate sensitivity to horizontal 
MPT impacts (integrated System Support concept) 

Sustainability 

MPT system must support manning/ training system(s) 
plans over long periods of time 

MPT system must consider extended impacts of mission/ 
scenario changes with time 


Most users expressed the idea that system design should be influenced by 
various manpower constraints emerging from early MPT analysis before the 
design is fixed. A related point is that most users thought that the primary 
tool for getting MPT projections into the system through the Quantitative and 
Qualitative Personnel Requirements Information (QQPRl) , which is developed 
concurrently with the Basis of Issue Plan (BOIP), was inadequate and/or 
untimely. By the time QQPRl data is available (after Milestone i), most of a 
system's design is fixed (Figure 4-1). Another problem mentioned was that 
DARCOM and TRADOC's shared responsibility for developing the QQPRl and BOIP 
requires additional clarification of lines of responsibility. This would be 
particularly important if the initial development of MPT projections shifts to 
an earlier point in the acquisition cycle (i.e., TRADOC would become 
responsible for the initial, therefore tentative, QQPRl). 

Occasionally, a user expressed a viewpoint that MPT projections made 
later in the acquisition cycle were adequate, providing that sufficient lead 
time was given to acquire and train the personnel. This statement assumed 
that funding was already available for the personnel. 
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1 11 111 

SYSTEM ACQUISITION LIFE CYCLE (MILESTONES) yearS 


SOURCE: Proceedings from National Security Industrial Association 

Symposium on NaVy Systems Acquisition, October 27*28, 1977. 


Figure 4-1. Effect of Design Decisions on Life-cycle System Cost 
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2. User Group Responses 

User responses are reviewed in this section. Responses from 
multiple users in DARCOM and SSC-NCR are incorporated under single headings 
for these respective organizations. Individual responses, by organization, 
are detailed in Appendix B. The discussion on user responses includes a 
summary of the user organization background to establish the appropriate 
context for the responses that follow. 


a. DARCOM/Integrated Logistic Support (ILS) Managers Responses . 
ILS managers in DARCOM are responsible for ensuring the planning and monitoring 
of logistic support elements for systems under development from Milestone I 
throughout the development process. The prime analytical tool used in ILS is 
Logistic Support Analysis (LSA). 

The LSA process consists of analysis efforts to completely define the 
support system and its interfaces with a materiel system. The analysis 
incorporates manpower/logistics analysis and tradeoff analyses associated with 
maintenance concepts (repair levels, etc.). LSA Record information (LSAR) 
includes all tasks required to operate and maintain a materiel system. LSA 
data then becomes feeder data for the Quantitative and Qualitative Personnel 
Requirements Information (QQPRI) system. As the chief means of developing 
both the numbers and kinds of personnel required for operating and maintaining 
systems, QQPRI development is a shared responsibility of DARCOM and TRADOC. 

As a result, LSA data needs to be as accurate as possible for both the DARCOM 
and the TRADOC user communities. 

LSA normally includes operator/maintainer task and supply functions for 
repair and test equipment as well as the system under development. In determin- 
ing system supportability , ILS managers required information about the exact 
tasks to operate and maintain systems. They also needed projections of reli- 
able manloading requirements to help establish expected Military Occupational 
Specialty (MOS) and skill demands, plus any MOS changes anticipated. 

ILS managers stated a need for methods to analyze tradeoffs between 
force structure, personnel constraints, hardware performance, and cost. 
Essential to this analysis was information (output) about support and test 
equipment ("playing field") associated with a system. The total QQPRI 
associated with an expected density of these systems, when deployed, was cited 
as an important element of this analysis. 

Early maintenance concept information was also requested, including 
maintenance manhours (MMH), reliability, availability and maintainability 
(RAM) data, and maintenance-level information. 

DARCOM suggestions for the design of an MPT model included providing/ 
displaying MPT data for alternative (proposed), as well as predecessor systems 
to provide support documentation for decision-making. An MPT methodology 
would need to provide credible quantitative data to the ILS process, the 
source(s) of which could be documented or traced. Formats for the data should 
be designed to present data efficiently to users who have limited time for 
reviewing information (e.g., graphs, charts, etc). Timing for initiating the 
process should be prior to Milestone I to achieve maximum benefit from 
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hardware /manpower tradeoffs; however, there should also be follow-up on 
existing systems in the field to verify LSAR data and system supportability 
information. 


b. SSC-NCR (TRADOC) Responses . SSC-NCR is responsible for 
assessing the feasibility of allocating personnel and materiel to specified 
organizations within the Army (i.e., the Tables of Organization and Equipment 
(TOE). SSC also ensures that the user communities identify both demand and 
supply MPT factors at the front end of the system acquisition process to 
address supportability and affordability issues. 

SSC-NCR stated that detailed data for the following general categories 
of information is required: 

( 1 ) Manpower . 

(2) Comprehensive task breakdown. 

(3) Personnel. 

(4) Training. 

(5) Logistic support. 

(6) Integrated System Support (ISS). 

Graphic display capability was highlightd as useful to SSC for inclusion 
in reports comparing demand and supply. In addition, detailed MPT analyses 
comparing proposed to predecessor systems is required. 

SSC also needed MPT projections prior to Milestone I to support the Army 
Systems Acquisition Review Council (ASARC) decision process. An additional 
need cited was timely and useful cost information for the Army's funding 
process. In providing such information, the projection process should also 
expose any assumptions employed so that an understanding of the limitations of 
the analysis is clear. 

SSC-NCR further suggested using a solid, historical data base containing 
Army personnel statistical data, comprehensive task-related files, and RAM 
data associated with systems. It was also indicated that the MPT system 
should be compatible with existing Army reporting structures and command roles 
to ensure that the model could be integrated into the Army's normal systems 
acquisition process. 


c. Training Developers (TRADOC) Responses . A key concern of 
training developers within TRADOC Headquarters is the integration of the 
Army's training function with the other critical Army functions - manning, 
force structure, facilities, equipping, sustaining, mobilizing, and 
deploying. Matching the Army training system to manning and the systems 
acquisition process is crucial to the support of force modernization. 
Supportability of force modernization requires specific information about the 
availability of manpower qualified for training, as well as sustainability 
(personnel flow and replacement) of the force required. 
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Another driver of training developers' needs for MPT information was 
system affordability, i.e., an understanding of a system's life-cycle cost. 
The general categories of information needed by training developers to 
determine supportability and affordability were (1) manpower and personnel 
information (demand/supply); (2) training information (exact operator/ 
maintainer task and target audiences); (3) equipment information (capability 
and man/machine tradeoffs); and (4) accurate maintenance concept information. 

d. Combat Developers/Soldier Development Directorate (SDD) and 
TRADOC) Responses . Combat developers in TRADOC/ Soldier Development 
Directorate are concerned with personnel doctrine that addresses air/land 
battle mission needs. They also promote combat effectiveness through MOS and 
force structure change and ensure that soldiers' physical and mental cap- 
abilities are taken into account by the various elements of force moderniza- 
tion. In their interface between SSC and TRADOC Headquarters, combat 
developers at SDD are involved in the process of optimization of resources 
between proponent schools. 

The combat developer's key drivers for needing MPT information were 
supportability , in a readiness sense; performance , as it relates to combat 
effectiveness (e.g., through man/machine design issues); and cost (e.g., as 
might be determined by impact analysis). 

Combat developers' design suggestions focused on the need for an 
interactive management information system having a time-line signaling 
mechanism that could alert participants to needed (or missing) actions in the 
current MPT process. Suggested outputs from the MPT model were a projection 
of the numbers and kinds of operators and maintainers associated with a 
system. A timely breakdown of task and training data was also cited to 
influence system design (prior to Milestone I). 

It was also suggested that some information concerning the impacts of 
new systems on the existing personnel structure and resulting tradeoffs would 
be useful in optimizing the selection of new systems. This requirement also 
implies a need to apply MPT analysis to systems on a horizontal plane (i.e., 
to consider the drain on the projected manpower pool caused by other develop- 
ing systems). 


e. Tradoc System Managers (TSM) Responses . During the 
gestation period of a system prior to Milestone I, TRADOC assumes the lead in 
conducting mission area studies. A TRADOC Systems Manager (TSM) is concerned 
with the identification, analysis, and selection of system concepts that meet 
needs of combat elements of the Justification of System Major New Starts 
(JSMNS) document. Other responsibilities of the TSM include preparing the 
Operational and Organization (0&0) plan, as well as monitoring the preparation 
of a number of different documents required for logistic support. Among these 
are (1) the Integrated Logistic Support Plan (ILSP); (2) the Concept Formula- 
tion Package (CFP); (3) the Outline of Individual and Collective Training Plan 
(OICTP); (4) the Letter of Agreement (LOA); (5) the Cost and Operational Effec- 
tive Analysis (COEA); and (6) the Preliminary Qualitative Assessment (PQA). The 
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Preliminary Quantitative and Qualitative Personnel Requirements Information 
(PQQPRI) could, by interpretation of the current regulation, be initiated by 
the TSM although it is not normally done at this time. 

The TSMs had a clear requirement for a model that would contain a com- 
prehensive task file data base with task data taken from a number of task- 
centered Army manuals and documents. This would assist the TSMs in ensuring 
the timely completion of these documents by facilitating the identification of 
specific tasks, MOS, and skills associated with hardware. Finally, the TSMs 
suggested the development of both a manpower and personnel (demand /supply) 
data base that would permit tradeoff analysis to determine the best fit of 
people with equipment. 


f. LOGCEN /LOGISTICS CENTER (TRADOC) Responses . Force 
developers, training, and materiel systems managers are all MPT actors within 
LOGCEN. The critical role of LOGCEN is as an executive agent for the ILS 
process. Issues such as man/machine interface, system and non-system training 
devices, logistic system proponency, and Manpower Authorization Criteria 
(MACRIT) are included in LOGCEN' s charter. Some of the LOGCEN responsibil- 
ities are (1) impact analysis of system/design characteristics; (2) support 
equipment training plan and technical publications development; and (3) MACRIT 
system improvement. As a TRADOC integrating center, LOGCEN is similar to a 
proponent school in its proponency role for selected logistic systems. LOGCEN 
also develops some TOEs for maintenance units. 

LOGCEN indicated they had specific information requirements related to 
system considerations for affordability and supportability , including 
(1) accurate RAM data (based on historical data and accounts for realistic 
utilization rates, actual failure rates, and combat damage); (2) workload data 
(as a function of direct productive and indirect productive annual maintenance 
manhours (AMMH), plus available productive manhours); (3) density of equipment 
(TOE-related) ; (4) unit-focused manpower requirements; and (5) LSAR-format 
task data associated with MOS and equipment needed for training analysis. 

LOGCEN' s MPT timing requirements were twofold: (1) LOGCEN needed early 

(front-end) training device information to prepare for both early system 
testing and the final system design configuration, as well as (2) solid 
manpower requirements for supporting accurate MOS and organizational design 
decisions . 

Design discussions about MPT projection models focused on LOGCEN' s 
requirement for accurate and reliable (RAM) data. The Annual Maintenance 
Manhour (AMMH) data that LOGCEN presently receives should be made separate and 
distinct from reliability data. MPT information was needed in an LSAR data 
format and would be more useful if broken down by component, task, and skill 
level. It was felt that this could then be used with an MPT simulation to 
determine impacts of utilization rate, combat damage, and/or hypothetical 
changes in RAM characteristics. 

A concern was expressed about the accuracy of data to be used for MPT 
analysis. The availability of "hard" LSAR data is critical to the development 
of useful MPT information. Given an absence of system fielding experience and 
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correspondingly mature LSAR data (such as would be the case when substantial 
technological changes in equipment are introduced), it was recognized that it 
would be difficult to obtain reliable data. However, it was also recognized 
that, if there is minimal technological change, the MPT information should be 
readily available. 


g. Devices and Systems Training Directorate (TRADQC) Responses . 
This organization is involved in policy development, training-device pro- 
ponency, and training-plan approval. 

In general, this directorate indicated needs for both manpower and 
training information for the purpose of developing the individual and 
collective training plans (ICTP) of systems under development. A new Army 
thrust called Integrated System Support (ISS) now requires the generation of 
MOS information that is system- as well as unit-specific. It was also 
suggested that an improvement to the current MPT process would be to provide 
sufficient front-end task manloading and training information to allow the 
design to be influenced early in the acquisition process. One final expressed 
requirement was to have an MPT projection model that more effectively flags 
MOS "overload" (i.e., a model that detects and documents additional needed 
manpower, both numbers and skills). 


h. Logistics Evaluation Agency - LEA (DCSLOG) Responses . As a 
field-operating agency of the DA staff, LEA is the independent logistician in 
the materiel acquisition process responsible for monitoring ILS requirements 
for systems under development. Supportability of the more than 800 systems to 
be evaluated was cited as a key driver for needing MPT information, along with 
other factors such as degree of interoperability and performance levels. 

LEA users indicated a need to be able to use MPT projections (before 
Milestone I) as constraints during system development. These MPT projections 
should include RAM goals , new skills and training device concepts. Following 
Milestone I, it was suggested that readiness considerations include a 3- to 
5-year lead time to acquire and prepare logistic support personnel prior to 
system fielding. LEA information requirements also included a complete and 
accurate listing of tasks and skills required to maintain the system 
(including all subsystems). Finally, there was an expressed requirement to 
look across systems (horizontal analysis) to assess overall impact on logistic 
support and training. 


i . Force Modernization Cell - Manpower Programs and Budget 
(DCSPER) Responses . This organization provides staff support to DCSPER for 
the Array Systems Acquisition Review Council (ASARC). They are chiefly 
concerned with resource management for force modernization and bridging demand 
and supply issues in both manning and force-structuring. DCSPER is concerned 
with matching the number and kinds of "spaces" needed to operate and maintain 
systems with the required qualified personnel (i.e., "faces"). In this 
respect, they are also concerned with the personnel training. Because of 
congress ionally-mandated "end strengths" for the Army, information concerning 
distribution of personnel is important. Recognizing that affordability and 
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user supportability concerning the supply of personnel is a realistic 
constraint, DCSPER therefore suggested that a horizontal look across systems 
would permit analysis of the impacts of shifts in the numbers and types of 
manpower required. Tradeoff analysis at a high level was also cited as useful 
for conducting Army-wide optimization studies (e.g., hardware vs. manpower vs. 
system performance). 

Finally, it was indicated that the design and output of an MPT model 
should be compatible with existing reporting processes (QQPRI, BOIP, ASARC 
Milestones, etc.). In line with this requirement, it was also suggested that 
there is a need for a more effective (interactive) information system, with 
appropriate access to relevant data bases of MPT information. 


j. Industry Responses . Three industrial associations 
(Aerospace Industries Association, Electronic Industries Association, and 
National Security Industrial Association) were surveyed about their needs for 
logistic support (MPT) information in responding to Requests for Proposals 
(RFPs). It was discovered that industry is very interested in improving its 
understanding of logistic support and human factors concepts associated with 
systems under development. Some suggested means to accomplish this were (1) 
earlier user contact, (2) user involvement in RFP preparation, (3) better 
implementation of human factors analysis, and (4) improved information 
dissemination to industry. Two basic concerns of industry have been to 
address human factors before the system design is fixed and to specify 
contract arrangements for implementing MPT analysis. 

Industry also indicated a need for reliable information concerning the 
intended/planned concept of system operation and supporting organizational 
design. The participants also suggested DOD strengthen its estimates of the 
kinds of personnel expected to operate and maintain the system. Scenarios 
depicting both threat, environment, and realistic usage rates were cited as 
important in preparing RFPs. The expected performance of a system, plus 
maintenance concept, were two other critical data points contributing to 
industry's ability to conduct or contribute to early MPT analysis. 

The industry-users interviewed also indicated a need for MPT information 
to be centrally located for accessibility. Charts and graphs were suggested 
as being especially helpful in transmitting to industry the necessary 
information. An historical data base was also suggested to permit comparisons 
with other developed systems or analysis of system designs that were rejected. 


2. Tabulation of Individual Interviews into Matrix Format 

After all user interviews were completed, the raw discussion notes 
were assembled, reviewed, and transcribed onto interview forms for clarity. 

Key words and ideas were highlighted. Two requirements analysts then 
separately analyzed the information and jointly tabulated the responses to 
facilitate analysis of the results. Once the answers were transformed into 
rough requirement statements, they were entered into a matrix format to permit 
a clear indication of which users were associated with each user need. The 
detailed matrix of users and user answers is shown in Appendix B. 
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D. TRANSLATION TO USER REQUIREMENTS 

1. Procedure 

The user needs were divided into two categories - one category 
containing "output" (data) needs, and the other containing "design" needs for 
an MPT projection system. 

User requirement statements were generated from the Appendix B matrix 
information and the interview forms. When appropriate, similar ideas or 
requirements expressed by different users were translated into a comprehensive 
user requirement statement. 

The separate requirement statements developed by both analysts were then 
compared to ensure agreement and completeness. To establish priorities among 
the requirements, criteria were developed to evaluate the requirements in the 
matrices. A final user-requirements list was developed after applying these 
criteria. 


2. Criteria for Assigning Priorities 

The following criteria were considered essential to determine 
which requirements were necessary or desirable for an MPT projection model: 

(1) The requirement's role in addressing key drivers associated 
with MPT projections for Army systems (i.e., affordability, 
supportability , and sustainability). 

(2) The importance of the requirement in providing useful and 
timely MPT information to the Army system acquisition 
process . 

(3) The breadth of users stating the requirement. 


3. Requirements List 

As stated earlier, the user needs were categorized by "output" and 
"design" needs and then prioritized according to the degree of fit with the 
three criteria stated previously. Once this procedure was completed, the user 
needs were grouped into mandatory and desirable requirements. Below is the 
final list of user requirements (including output and design requirements) for 
a projection system that conforms to the needs of the key Army players in the 
MPT arena. For the purpose of this evaluation, all the following mandatory 
requirements were given equal weight because, ultimately, they were primarily 
used to test the "reasonableness" of the methodology: 

(1) Mandatory Output Requirements: 

(a) Operator/Maintainer task data should be provided as 
required for a system and its support equipment 
(including task description and frequencies), based on 
a replaced or comparable system. 
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(b) Maintenance manhour projections must be based on a 
specified maintenance concept (levels of maintenance, 
specific RAM data). 

(c) Operation and maintenance manpower, personnel, and 
training (MPT) projections must reflect realistic 
system usage rates as determined by acquisition- 
specific mission/scenario drivers. 

(d) Reasonably accurate (i.e., acceptable to the 
budget-control community) total manpower demand 
(number, MOS, skill level, additional skill 
indicators) is required before Milestone I. 

(e) The MPT model must include and document projected 
availability of personnel for the system (demographic 
data such as number, age, sex, size, skills, 
education, MOS, term of service). 

(f) The MPT model should document MOS alignment data 
showing current status; authorizations vs. 
requirements to indicate MOS overloads. 

(g) The model must incorporate, document, and use target 
audience description (TAD). This includes reading 
ability, grade levels, experience, skills, and 
previous training of available personnel to determine 
accurately testing/training requirements (the TAD 
would be input to the Letter of Agreement). 

(h) The "T” of MPT must include the formulation, descrip- 

tion and use of comprehensive training concept 
elements: (1) classes or training required (including 

any increases) for unit or institutional training; 

(2) time to train; (3) training frequency; and 
(4) training support (devices, facilities, 
instructors). 

(i) There must be documentation of MPT impacts and costs 
on the Army as a whole. 

(j) Trade-offs must be conducted between hardware and 
people to determine the optimal balance for best 
system performance, costs (LCC), and support 
requirements (Decision/Tradeoff matrix needed). 

(k) Output should include reports with well-labeled, 
easy-to-read graphic formats such as graphs, tables, 
matrices, charts, as well as an executive summary of 
the report that could stand alone. 

(l) Outputs should be in both hardcopy as well as on CRT 
display for permanent archiving. 
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(m) Reporting procedure must include any pertinent caveats 
about the data source availability/reliability or 
assumptions about the system. This information should 
be provided at major decision points or steps in the 
analysis. Appropriate actions to resolve identified 
weaknesses in the analysis should also be noted. 

(2) Desirable Output Requirements: 

(a) A complete historical MPT data base is highly desir- 
able. MPT projections should be updated throughout 
the life of the system to increase accuracy of LSA/ 
other data. 

(b) MPT impacts should account for geographical constraints 
on manpower pools "available" to run the system. 

(c) MPT system output should be compatible with accepted 
LSA reporting format. 

(3) Mandatory Design Requirements: 

(a) Projected MPT impacts/costs must be available at the 
front end of the acquisition cycle (before Milestone 
I) for early design influence and early MPT (cost) 
warnings . 

(b) Front-end determination of a system's candidacy for 
MPT analysis should include criteria such as the 
potential for high payoff (e.g., selection should be 
based on type and amount of change in the technology 
that drive significant MPT costs/changes). 

(c) To facilitate training analysis and to support 
proponency for MOS changes or additions, the system 
should have the capability to compare the proposed 
operator and raaintainer tasks associated with a new 
system with a comprehensive, automated data base of 
tasks detailed in the Soldier's Manual, ARTEP, POI, 

SQT, T.G. and AR 611-201 (Enlisted Career MOS 
Management Fields). 

(d) Output should include comparable MPT projection data 
for both alternative/proposed and existing systems as 
support for decision milestones. 

(e) The model should incorporate horizontal analysis 
across systems to determine total MPT and logistic 
impacts and to facilitate Army-wide equipment and 
personnel optimization, as well as support 
end-strength decisions. 

(f) The MPT model should provide compatability with 
existing DOD documentation and reporting structures. 

It should, however, evolve into an interactive 


4-14 



information system with remote terminals, including a 
time line oriented to both the budget and systems 
acquisition processes. Signalling mechanisms are 
needed for late/missing/incomplete data. 

(g) The MPT system must identify and provide resolutions 
to both weaknesses in data inputs and inaccuracies 
resulting from the use of the data. 

(h) There must be a documented and accessible audit trail. 

(i) A universal information system should be designed to 
allow multi-user access and update. 

(4) Desirable Design Requirements: 

A more detailed 0&0 plan is needed to act as a "hingepin" to 
initiate MPT analysis at the front end of the acquisition 
cycle. 


E. RELEVANCE OF USER REQUIREMENTS TO EXISTING HARDMAN DESIGN 

Having stated the user requrements in the preceding section, it is now 
possible to determine the extent to which the HCM has application to the 
requirements for the Army's existing (or near-term) MPT process. Through a 
comparison of the HCM to user requirements, it was possible to establish one 
measure of the face validity of HARDMAN (i.e., how well the HCM fits the 
Army's MPT needs). This was achieved by checking the relative applicability 
of each step in the HCM to specific portions of the requirements. As stated 
earlier, both the ARI replication exercise and the HARDMAN applications audit 
were conducted in parallel with the requirements analysis. These parallel 
studies afforded an in-depth understanding of each step in the methodology. 

The additional familiarization with the complete array of data inputs, 
algorithms, methodology logic process, and outputs, provided the essential 
background to perform a comparison. Table 4-3 presents the results of the 
compatibility analysis by indicating which portions of the HCM have high, 
medium, or low applicability to the user requirements. The degree of 
applicability provided an indication of the utility of various portions of the 
technique to the Army's MPT users. 

Out of the thirteen total mandatory output requirements, the HCM 
demonstrated good compliance in essentially six of the user need areas. Of 
the seven requirements where elements of the methodology appeared to exhibit 
low applicability, only five of the user areas were considered of immediate 
importance to developing a sound MPT projection scheme. These key user areas 
were (1) establishing the accuracy of the MPT methodology; (2) including other 
demographic data such as age, sex, size, etc., in personnel allocations; 

(3) considering the demands of possible future acquisitions on MOS, (4) per- 
forming a complete life-cycle cost analysis as part of the tradeoff studies; 
and (5) indicating weaknesses in the analysis and respective resolution 
actions. Items (2) and (3) appear to be somewhat of an Army-wide constraint 
resulting from the general lack of readily available information. Therefore, 
only requirements (1), (4), and (5) seem to be achievable, close-range 
deficiencies that should be incorporated as improvements to the HCM. 
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Table 4-3. Comparison of User Requirements to the 
HARDMAN Comparability Methodology 


User Requirements 


Applicability of HCM Technique 
to User Requirements 


High Medium Low 


Manadatory Output Requirements 

Operator/maintainer task data as required 
for system and support equipment 
( description/ frequencies ) based on 
replaced or comparable system 

Maintenance manhour projections must be 
based on specified maintenance concept 
(levels of maintenance, accuracte RAM 
data) 

Operation/maintenance requirements must 
reflect realistic system usage rate as 
determined by mission/scenario drivers 

Reasonably accurate total manpower demand 
estimate (number, MOS, skill level, addi- 
tional skill indicators) required before 
Milestone 1 

Model must include and document projected 
availability of personnel for the system 
(Demographic data such as - number, age, 
sex, size, skills, education, MOS, term 
of service) 


Model should document MOS alignment data 
showing current status; authorizations 
vs. requirements to indicate MOS overload 


Model must incorporate and document target 
audience description - including reading, 
grade levels, experience, skills, previous 
training of available personnel to accu- 
rately determine testing/training require- 
ments (would be input to Letter of 
Agreement) 


X X 

(Primary (Support 
System) Equipment) 

X 


X 


X 

(Manpower 

demand) 


X 

(No., 
skills , 
MOS, 

Term of 
service) 

X* 

(Current 
system 
demands ) 


X 

(Model 

assumes 

prede- 

cessor 

training 

is 

adequate 


? 

(Accuracy 

unknown) 


X 

(Age/sex/ 

size 

education) 


X 

(Future, 

multi- 

system 

demands) 
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Table 4-3. (Cont'd.) 


User Requirements 

Applicability of HCM Technique 
to User Requirements 


High 

Medium 

Low 

Manadatory Output Requirements 




The "T" of MPT must include the formula- 
tion, description and utilization of 
comprehensive training concept elements: 
classes required (including any increases) 
for individual; unit or institutional 
training: time to train; training 

frequency; and training support (devices, 
facilities, instructors) 

X 

( Insti- 
tutional 
train- 
ing 


? 

(Collec- 
tive unit 
training) 

Documentation of MPT impacts and associ- 
ated costs for MPT projection 

X 



Tradeoffs between hardware and people 
needed to determine optimal balance 
between performance: costs (LCC) and 

support requirements (decision/tradeoff 
matrix needed) 


X 

(Limited 

tradeoff 

capability) 

X 

LCC and 
other O&S 
elements) 

Output should include well-labeled, easy- 
to-read formats such as graphs, tables, 
matrices, charts, as well as an Executive 
Summary of the report that can stand 
alone 


X 

(Executive 
Summary ) 


Outputs should be in hardcopy as well as 
CRT display for permanent archiving 


X 

(Hardcopy) 

X 

(CRT) 

Reporting procedure must include briefing 
re: any pertinent caveats about the data 

source availability/reliability or 
assumptions about the system, at major 
decision points or steps in the analysis. 
Actions should be noted to resolve 
weaknesses identified 


X 

(Not consis- 
tently done) 

X 

(Resolu- 
tion 
actions 
on major 
weak- 
nesses ) 
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Table 4-3. (Cont'd.) 


User Requirements 

Applicability of HCM Technique 
to User Requirements 

High Medium Low 

Desirable Output Requirements 



Historical MPT database highly desirable. 

X 

X 

MPT projections should be updated through- 

(Histor- 

(Update 

out life of system to increase accuracy of 

ical 

over 

LSA/other data 

database) 

system 

life) 

MPT impacts should account for geo 
graphic constraints on M.P. pools "avail- 
able" to run the system 


X 

MPT system output should be compatible 
with accepted LSA reporting format 

Mandatory Design Requirements 


X 

Projected MPT impacts/costs must be avail- 
able at the front end of acquisition cycle 
(before Milestone I) for (a) design 
influence and (b) early MPT (cost) 
warnings 

X 


Front-end determination of system's candi- 
dacy for MPT analysis should include 
criteria such as potential for high 
payoff, e.g., selection should be made on 
type and amount of change in the technol- 

X 


ogy that mightttdrive significant MPT 
costs/changes 



To facilitate training analysis and 

X 

X 

proponency for MOS changes or additions, 

(New 

(Auto- 

the model should have the capability to 

skill 

mation) 

compare operator and raaintainer tasks 

identif i- 


associated with a selected system with a 
comprehensive automated database of tasks 
detailed in the Soldier's Manual, ARTEP, 
POI, SQT, T.E . and AR 611-201 (enlisted 
carreer-MOS Management Fileds) 

cation) 
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Table 4-3. (Cont'd.) 


User Requirements 

Applicability of HCM Technique 
to User Requirements 


High Medium Low 

Mandatory Design Requirements 



Output should include comparable MPT 
projection data for alternative (proposed) 
and existing systems as support for 
decision milestones 

X 


Should incorporate horizontal analysis 
across systems to determine total MPT and 
logistic support to facilitate ARMY-wide 
optimization and support end strength 
decis ions 


X 

MPT model should provide compatibility 
with existing DOD documentation and 
reporting structures; however, it should 
evolve into an interactive information 
system with remote terminals, including a 
time line oriented to both the budget and 
systems acquisition processes. Signaling 
mechanisms are needed for late/missing/ 
incomplete data 

X 

(Exist- 
ing docu- 
menta- 
tion) 

X 

(Remote 

terminals 

with 

time-line 

overlay) 

MPT system must identify/resolve weak- 
nesses in data inputs and resulting 
analysis 


X 

Documented and accessible audit trail is 
essential 

X 


Universal information system (CDB) should 
be included and designed to allow multi- 
user access and update 

X 

(Consol- 

idated 

data- 

base) 

X 

(User 

access) 

Desirable Design Requirements 



More detailed 0&0 Plan required as 
"hingepin" to initiate MPT analysis at 
front end 


X 

(Army 

responsi- 

bility) 
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Out of the nine mandatory design requirements, the technique seemed to 
be in reasonable agreement with seven of the user needs. The two areas where 
there was low agreement were considered critical for the near term. These two 
areas were, (1) consideration of the horizontal effects of other systems on 
the manpower drain and (2) construction of a means to flag and resolve 
weaknesses in the MPT analysis. Both Items (1) and (2) were already discussed 
as being an equally important consideration on the "output" side of the system. 

Overall, this discussion suggests that the HCM is reasonably applicable 
to the Army's major user needs in its present design. More specifically, the 
present HCM basically meets seventeen of the twenty-two crucial user needs. 
Therefore, by being largely commensurate with the present user requirements, 
the technique seems to have considerable near-term utility to the Army. 
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SECTION V 


OPERATIONAL ANALYSIS 


A. OVERVIEW 

Section IV provided one measure of face validity by indicating the 
relative compliance of the HCM with independently-derived user requirements. 
This part of the analysis suggested that the methodology had considerable 
utility to the Army. Other important face-validity considerations were 
introduced in Section III. These considerations refer more to the general 
integrity of the technique and include such elements as potential error 
sources, logic, reasonableness or credibility of the projected results, 
inherent ambiguities in the logic or outcome, and how supportable the model is 
by its respective input data sources. To close out these last elements of the 
f ace-validity analysis, a multi-leveled approach was structured, basically 
consisting of (1) a comparison of the HCM against any MPT modeling schemes 
delineated in existing DOD and MIL-STD instructions; (2) a comparison of the 
HCM against a host of other known MPT projection or modeling schemes; and 
(3) an independent, detailed, step-by-step audit of actual applications of the 
technique. This part of the overall study was termed "operational analysis" 
because every aspect of the methodology was actually exercised quite 
rigorously. The following paragraphs first provide the detailed rationale and 
approach used to investigate the various remaining face validity issues, 
followed by the actual analysis and conclusions concerning the "general" 
integrity of the HARDMAN system. 

B. APPROACH 

To ensure that the HCM was generally sound, beyond the user requirements 
compliance, several additional face-validity elements were considered. The 
additional face-validity elements and respective tests were as follows: 

(1) Logic - A simple examination of whether the proposed technique 
maintains a well-structured, credible train of logic throughout 
the model . 

(2) Usability - A simple study of both the utility of the model and 
the ease with which the model can be applied by others. 

(3) Error Sources - An investigation of whether the methodology 
incorporates data inputs or internal procedures that can cause 
inaccuracies in the projections. 

(4) Data Availability - A basic check on whether the modeling 
technique draws on real-world, available data sources to act as a 
foundation for the projections. 
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(5) Ambiguity - An examination of the ability of the model to be both 
consistent and succinct in its respective use of data and overall 
design (e.g., having each successive step build on previous 
steps). The ambiguity test also includes a look at how clearly 
the results relate to the model's original objective(s) . 

(6) Credibility - A simple test of whether the results seem 
reasonable, based on the inputs and whether the technique 
incorporates a means of weighing the quality of the data inputs 
and results. 

A three-phased analysis was structured within the context of the 
preceding face-validity tests. The first and second phases of the analysis 
primarily addressed the logic, usability, error source, and data-availability 
elements. The third phase of the face-validity analysis encompassed all six 
of the above elements. 

As an initial step in evaluating any new modeling scheme, it is useful 
to compare the new methodology against similar techniques that appear 
reasonable and/or are already tested and in use. This comparison serves 
several purposes in the context of the above first four validity elements. 
First, it provides a basis to examine whether the overall logic and structure 
of the model are reasonable. Second, the comparison establishes a yardstick 
against which to measure improvements in the versatility and subsequent 
utility of the model. Third, it assists in flushing out inherent sources of 
errors. Last, the comparison sheds light on the availability and quality of 
data. 


To these ends, the first phase of the validity analysis examined 
existing DOD and MIL- STD guidelines to see if any credible MPT projection 
schemes have already been established. Similarly, the second phase consisted 
of conducting a much larger literature search inclusive of past and present 
MPT models from both the government and private sectors. 

The third phase of the face validity study was a detailed audit of three 
applications of the projection technique. This part of the operational 
analysis proved to be a pivotal component because it encompassed all the 
validity insights provided by the above literature searches , as well as a 
grasp of (1) the ease with which the model could be applied by others; (2) 
whether the technique contained discontinuities between its inputs, internal 
mechanics and results (therefore introducing ambiguities); and (3) whether 
there was a clear indication of the quality and limitation of the outcomes. 

The three applications chosen for the audit were the Sing le Channel Ground 
Airborne Radio System (SINCGARS), Remotely Piloted Vehicle (RPV), and the 
Division jJupport Weapons System (DSWS). The results of the three remaining 
phases of the face-validity analysis are detailed in the following paragraphs. 
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c. 


ANALYSIS OF EXISTING DOD MPT PROJECTION GUIDELINES 


As indicated in Section II, there are several DOD and MIL-STD documents 
that refer to MPT. For example, DOD 5000.1 and 5000.2 describe the structure 
and supporting procedures for planning and completing major system acquisi- 
tions (1,2). As a precursor to 5000.2, the 5000.1 directive calls out the 
necessity to define, estimate, and budget accordingly, MPT at the acquisition 
management level. The 5000.2 directive, acting in a supplementary role, more 
specifically addresses how the new acquisition will be evaluated against the 
various support parameters such as MPT. For example, using a top-down 
approach, it first defines the acquisition review council (DSARC or ASARC) 
advisory roles and then establishes both the content and approximate time 
frames for the major acquisition and system development review phases. From 
the standpoint of threat mitigation, affordability, and supportability , 
detailed guidelines are provided for developing the system acquisition and 
support strategy. The 5000.2 directive states the necessity to consider MPT 
needs under the "System Readiness, Support, and Personnel (ILS)" portion of 
the document. Although this subsection requires that MPT estimates consider 
both peacetime and wartime manpower demands, skills, costs, and training, it 
does not indicate a methodology to assist in making these estimates. 

Similarly, the 5000.1 directive provides even less distinct guidelines for 
determining, and budgeting for, MPT costs. Because MPT falls under the 
heading of Integrated Logistics Support (ILS), the next step was to examine 
directives more specifically related to system support. Perhaps one of the 
best of the family of directives associated with system support is MIL-STD 
1388A (11). This MIL-STD states up front the requirement to develop human 
engineering and safety programs as part of the overall system support analysis 
plan. The document then proceeds to delineate key MPT variables and sup- 
porting data that must be considered to determine whether the proposed system 
meets manpower readiness, supportability, and affordability goals. These 
variables include (1) the number of systems to be supported; (2) missions per 
unit of time; (3) mission duration; (4) operating parameters (days, miles, 
hours, flights, etc.); (5) maintenance and transportation periods; (6) des- 
criptions of system components and operating environment (in terms of their 
impact on MPT); (7) task descriptions; (8) costs; and (9) special support and 
test equipment MPT impacts. Additionally, MIL-STD 1388A requires that all the 
various support scenarios for each proposed design alternative be compared 
against each other to optimize the design selection. 

As in the case of the previously described DOD directives, MIL-STD 1388 
also fell short of providing an analytic structure for projecting new 
acquisition MPT demands. Although a dead-end from the analytic standpoint, 
MIL-STD 1388A did provide more depth on some positive characteristics of the 
HCM. These characteristics revolved around the reasonably close match between 
the DOD MPT guidelines and structure of the data inputs and outputs of the 
HCM. Because MIL-STD 1388 provided more specific guidelines on MPT support 
considerations than DOD 5000.1 and 5000.2, it was decided to probe one level 
deeper into specific Army directives having to do with MPT. At this final 
level it was anticipated that all key MPT variables and data elements would be 
defined along with the supporting analytic structure. Probably the most 
concise Army directive in the MPT area is Army Regulation (AR) 570-2 (12). 
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This service regulation, entitled the "TOE Manpower Authorization Criteria 
(MACRIT)," provides criteria and standards that help establish an "equitable 
relationship between services performed and types of personnel utilized." To 
achieve this match the MACRIT requires that system manpower studies be 
conducted and the results approved by the Deputy Chief of Staff for Personnel 
(DCSPER) for the Army. The studies must provide the number of direct workers 
necessary to effectively perform a specified work activity. Direct (actual 
task performance), nonproductive (sleep), and indirect (travel and task 
preparation) times must be considered in the manpower calculation. The MACRIT 
defines sane approximate time factors for these three elements and ultimately 
provides a simple equation to calculate manpower. This equation is as follows 
( 12 ): 


WORKLOAD FORCE STRUCTURE 

(in direct and X (in number of 

indirect manhours per system) systems to be procured ) 

AVAILABLE PRODUCTIVE HOURS 


MANPOWER 
(number of 
people) 


Figure 5-1 shows that the above expression can be depicted graphically 
as a rectangular function. 

One interesting aspect of the MACRIT manpower equation is that the 
number of support people is calculated for the complete procurement totally 
independent of the acquisition time cycle. Therefore, if the manpower require- 
ment per year was graphed as a function of the system life cycle, one would 
observe the same type of step function as shown in Figure 5-1, indicating no 
change in the manpower requirements over time. This function is discussed in 
greater detail later in this section. The most important aspect of the above 
discussion is that AR570-2 presently represents the key manpower projection 
and planning document in the Army. It is now also obvious, based on the HCM 
sunmary provided in Section II-B, that the methodology is strongly aligned 
with the manpower guidelines as specified in the MACRIT, as well as the 


MANPOWER 

PER 

SYSTEM 




NUMBER OF SYSTEMS 


Figure 5-1. MACRIT Manpower Curve 
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broader policy guidelines spelled out in major support and acquisition 
planning documents such as MIL-STD 1388A, DOD 5000.1 and DOD 5000.2. 

It is clear from this discussion that related DOD directives do not seem 
to provide much analytic guidance in the area of new acquisition MPT projec- 
tions. Although one appraisal element was obtained via the MACRIT, this data 
point only represented one aspect of evaluating a much more complicated metho- 
dology encompassing task and skill analysis, personnel identification, training 
identification, costing, and tradeoff analysis. To obtain more data points on 
the manpower algorithm and evaluate the other methodology elements, it was 
obvious that a broader literature search was required. The following section 
provides the results of this broader effort. 


D. CRITIQUE OF OTHER EXAMPLE MPT PROJECTION METHODS 

The summary of directives provided in the previous discussion was not 
meant to be an exhaustive review of all related MPT DOD documentation. For 
the purpose of this study it sufficed to point out that the immediate and most 
widely used DOD acquisition and support guidelines are not fruitful sources of 
detailed MPT modeling techniques. A somewhat similar approach was taken with 
the literture search. Again, the intent was not to cover all modeling 
developments in MPT. For this study it was adequate to examine a sufficient 
number of related papers and models to obtain a solid understanding of other 
accepted techniques and logic employed in government and the private sector, 
compare shortcomings of both the DRC and other models, and ensure that a large 
enough array of models was examined so that all aspects of the HARDMAN 
methodology could be evaluated. This approach turned out to be very efficient 
because the literature search revealed a considerable amount of redundancy in 
the MPT material. In line with this finding, the following summary and 
analysis encompass a select group of papers that appeared to have the best 
data and analytic foundations as related to their respective aspect of the MPT 
problems. As stated earlier, the HCM covers a broad number of MPT topics. 
Therefore, to prevent any confusion in understanding how the various papers 
apply to the different topics, the following discussion is organized by each 
step in the methodology. The methodology steps described earlier in Section 
II-D are restated here in a little more depth to assist the reader in seeing 
the distinctions, or parallels, between the different projection techniques. 


1. Summary of Papers Related to Building the HCM Historical Data Base 
(Step 1) 

Step 1 of the HCM requires the development of a reference system 
that is representative of the proposed system (meets the functional require- 
ments) and is generally composed of technically and operationally similar 
predecessor components. The development of the reference system is a pivotal 
point in the MPT analysis because a conceptual error in system configuration 
could result in gross MPT errors later in the analysis. The premise for 
conceptually building the proposed system out of predecessor components is 
based on the idea that technology growth is usually evolutionary rather than 
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revolutionary in nature. Two references were discovered, Fong's unmanned 
spacecraft cost model and Gordon's review of mining equipment R&D, that 
illustrate the "evolutionary" concept of technology development as being a 
reasonable and supportable assumption (13,14). Both the Fong and Gordon 
papers were chosen to show that the accepted concept of evolutionary 
technology development in fact spans several industries (i.e., the 
aerospace/ communications industry to the mining industry). 


2. Summary of Information Related to the Manpower Algorithm (Step 2) 

As stated earlier in this section, one type of manpower projection 
algorithm is the simple Army MACRIT form (12). This form is the one employed 
in Step 2 of the HCM where the mission, operational, and maintenance data 
(extracted from the proposed and adjusted predecessor system data bases), are 
combined to establish the new system manpower requirements. Quade and 
Boucher, in their book entitled Systems Analysis and Policy Planning , employ 
the same basic rectangular function for projecting weapon system manpower 
support (15). They suggest that this algorithm is desirable largely because 
it provides a simple approximation of manpower (15). Another interesting 
insight, somewhat in conflict with the Quade and Boucher approach, is provided 
by Putnam in his paper dealing with projecting manpower to operate a new 
computer software system (16). Putnam provides essential reference backup 
supporting the theory that medium- to- large sized systems follow a life-cycle 
pattern of (1) rise in manpower, (2) peaking, and (3) gradual tailing off as 
the system gets older and is attrited. This cycle is termed the Norden/ 
Raleigh cycle and closely approximates a critically damped oscillating system 
or a Weibull curve as shown in Figure 5-2. 



P 

t = LIFE CYCLE IN MONTHS OR YEARS 


Figure 5-2. Typical Norden/Ral eigh Manpower Curve 
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The equation, depending on the boundary conditions and how gradually the curve 
peaks and tapers off, is of the form: 


MP = C x + C 



or, MP = C^+ C^t 


-bt 

e 


2 


where Cj, C 2 > and b are constants determined by the boundary conditions 
MP = 0 @ t = 0, dMP/dt = 0 @ t = tp (t peak), MP = MP max @ t = t p 
(t peak). The total manpower required for the life of the system is 
determined by integrating the above funtion(s) over the total life cycle. The 
important thrust of this paper is not determining the exact MP equation, but 
the fact that the author provides substantial backup indicating that small, 
short-lived projects can be modeled fairly accurately, using a rectangular MP 
function (16). However, large, long-lived programs (such as a large-weapon 
system) may require a manpower curve reflecting the more accurate buildup and 
tapering off of real-life systems (16). This could have major impacts on 
early- or later-stage, life-cycle MP requirements because the rectangular 
function would show a larger requirement than actually needed. Consequently, 
a new system being jointly phased-in at the same time or at a later stage 
might not have access to a needed manpower pool if the rectangular curve were 
used as an acquisition planning guide. 


3. Summary of Papers Related to the HCM Training Resource Assessment 
(Step 3) 

Step 3 of the HCM addresses the determination of training 
resources. Step 2 of the procedure provides both the number and types 
(skills) of people required for the proposed systems. Drawing on this 
information, simple programmatic matchups based on existing schools and 
training algorithms are employed to calculate the kinds and number of courses, 
instructors, training devices, and total training costs. The approach uses 
present curricula, devices, and associated costs where predecessor system 
components, similar to the proposed system, already exist. However, where new 
skills and training may be required the methodology provides logic algorithms 
(in the form of decision trees) for (1) determining whether additional 
training is required and (2) whether (and what kinds of) generic training 
devices are required. Ballpark cost estimates are then developed by using 
present costs of the generic devices as they exist in the other services or 
industry. 

The PACER document, introduced earlier in Section II, is almost 
uniformly in consonance with the HCM (7). One effort derived from the HCM and 
developed by Winston, "A Preliminary Report on the Early Comparability 
Analysis (ECA) Methodology," examines the difficult problem of identifying, 
focusing in on, and training for, new hard-to-find skills (17). 
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Winston's effort primarily addresses the issues of force modernization 
versus the recruitment/retention of manpower. The object of the comparability 
analysis is to identify potentially high MPT drivers early in the acquisition 
process. Data on specific problem tasks or "high drivers" would be supported 
by historical, current, or projected data on MPT. Examples of data sources 
include (1) Army Occupational Survey Program (AOSP), (2) proponent schools 
(TRADOC), (3) subject matter experts, (4) Logistic Support Analysis Records 
(LSAR), and (5) Army Publications. The basic methodology involves the 
following steps: 

(1) List the required tasks and MOS for the proposed system by using a 
similar predecessor. 

(2) Establish task criteria such as % of people performing a task 
(PPT), task difficulty (TD), frequency of task performance (FTP), 
risk involved if task not performed properly (Rl), task learning 
difficulty (TLD), training time (TT), and task decay rate (TDR). 

(3) Assign subjective values of 1-4 (where 1 is "low" and 4 is "high") 
to each criterion, for every task identified with the proposed 
system. 

(4) Calculate a subjective task score for each task (i.e., PPT x TD x 
FTP x RI x TLD x TT x TDR = Task Score). 

(5) Identify high drivers. 

(6) Conduct task analysis of high drivers. 

(7) Establish learning/training requirements for high drivers. 

(8) Determine personnel required based on test scores and available 
skills . 

The logic algorithms employed in the HCM decision tree analysis of new skills 
use the same subjective task criteria as listed in Step 2 of the preceding 
comparability methodology. 


4. Summary of Papers Related to the HCM Personnel Requirements 
Analysis (Step 4) 

Step 4 of the HCM establishes the necessary number of people in 
the training pipeline to support the manpower and skill projections made in 
Step 2. The pipeline figure considers that personnel move in and out of 
various skill categories for reasons such as retirement, promotion, or 
changing to different skill areas. The fractional change in people moving in 
and out of various skill areas is based on historical data. 
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Several papers associated with modeling this flow process were found. 
Driven by a desired manpower ceiling for various skills, each model basically 
establishes the monthly or yearly proportional ebb and flow of people within 
each skill category (based on historical data) and compares the final remain- 
ing stock of people in each skill area against the desired ceiling to estab- 
lish manpower shortages. Papers by Enke (18), Holz (19), Safeer (20), and 
Doong (21), apply specifically to the military (i.e., Army and Navy); while 
papers by Drummond (22), and Leeson (23), examine the problems of statewide 
agricultural labor shortages and police shortages, respectively. Although the 
terms in the equations describing the personnel flow process vary from paper 
to paper, the general algorithms are the same. The algorithms have the 
following basic form: 


V (mp . ) . = v p. (i - (f - f TM + f)). 

/ . A t l OM IH R l 

i=l i=l 


mp e - jr (MP c ) r jr («Vi 

i=i i=i 

where 

(MP^)^ = Manpower available in the ith skill category after reducing 

1 the existing pool 

P^ = Existing or prjected pool of people in the ith skill category 

f^ = Fraction of people out-migrating from skill category i 

OM 

f IM = Fraction of people in-migrating 

f = Fraction of people retiring 

R 

(MP ) . = Desired manpower ceiling for the ith skill category 

v 1 

MPg = Total entrant manpower required to sustain the desired 
manpower ceiling 

Enke suggests using probabilities in place of the fractional adjustments of 
the manpower pool to determine the possibility of joint events occurring 
(e.g., the chances of an individual leaving the Army within the interval t + n 
given that he has attained a certain level of training in a given skill area) 
(18). Enke further indicates that variables such as the state of the economy, 


5-9 



whether there is cr it not a draft, or massive reorganizations can effect the 
personnel flux greatly (18). Of all the papers, the most satisfactory approach 
to dealing with these variables is provided by Enke. Enke points out that 
historical manpower flows are only useful if (1) stable socio-economic condi- 
tions have existed and will continue to exist in the near future, (2) there is 
enough cause-effect data to catch most reasons for reenlistment/loss , and 
(3) the cause-effect data are supportable by accurate statistics. However, to 
gain experience on how historical manpower flows are perturbed by, for example, 
a varying economy, he suggests using the Decisions Elements Analysis (DEA) 
technique (18). DEA is a survey approach whereby statistically significant 
sample populations of personnel enlisting, leaving, and staying are queried to 
establish the main drivers affecting their respective decisions. Modeled 
after similar successful consumer product-oriented sales DEAs, this approach 
provides a more accurate picture of how the population will largely behave 
during periods of rapid, sizeable socio-economic change. The different splits 
in the sample populations are then used to proportionally adjust the pools 
within various skill areas so that the true manpower availability more closely 
resembles the real world. Enke anticipates that, with time, sufficient 
experience with widely varying socio-economic conditions can be gathered so 
that personnel flow models can employ different sets of probabilities as they 
apply to the given (or projected) national or international situation. 

By comparison, the HCM differs in the use of a starting manpower pool 
and desired ceiling. Both of these variables are required data inputs to all 
of the previous models examined. The HCM calculates the desired manpower 
ceiling from Step 2 of the methodology and does not depend on a historical 
starting pool. Instead, the HCM uses an optimization routine initiated by 
placing one hypothetical individual (unity) in the first open-skill area 
associated with a given task. The value of unity is multiplied by the 
historical fractional manpower flows and what initially remains is a fraction 
of an individual. The model then proceeds to input more individuals in the 
same manner, stepwise, until all the skill areas are filled in accordance with 
the manpower requirement. This stepwise calculation results in the total 
number of people in the pipeline that must be recruited and trained to 
compensate for the various personnel losses. The HCM does not appear to 
address the use of compensating factors to adjust its historical personnel 
loss rates according to socio-economic fluctuations. 


5. Summary of Papers Related to the HCM Determination of Manpower 
Shortages (Step 5) 

Step 5 of the HCM draws on the results of the personnel flow 
analysis and ultimately makes a comparison between the pipeline requirements 
and the present (or two-year projection) of available manpower. The resultant 
difference indicates manpower shortages in the various skill areas. The key 
difficulty with this step is finding a reasonable estimate of available 
manpower. If the manpower shortage is required for a near-term assessment, 
then the existing manpower pool is a reliable number. However, if the 
potential shortages must be calculated for a system deployed five to ten years 
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from now, then the current (or near-future) availability number may be 
inadequate. Several papers address the long-term manpower projection 
problem. All of the studies (Lindsay's future Army enlistment regression 
(24), Barnes' soldier 90 regression (25), and Grissmer's year 2000 manpower 
supply scenario (26)) were service-specific. Both the Lindsay and Barnes 
papers were standard linear regression models that project outyear manpower 
requirements. Grissmer employed a non-linear regression model that further 
adjusted its manpower trends up or down through the use of elasticities (i.e., 
weighting factors such as declining economy, unemployment, pay changes, etc.), 
which are projected to take effect at various time intervals and therefore 
distort the curve up or down as they occur. The major drawback of this 
techniques is that adjustments are not made to account for the lower 
confidence associated with outyear projections. 


6. Summary of Papers Related to the HCM Tradeoff Analysis (Step 6) 

The last step in the HCM is reflective in that various "what if" 
scenarios are developed in response to excessive costs or manpower and 
training shortages revealed by the results of Step 5. Many of the "what if" 
scenarios revolve around cost because it is a common denominator for many 
design decisions. In performing the various tradeoff analyses as a function 
of cost, the HCM usually does not extend beyond those elements directly 
associated with MPT (e.g., tradeoffs related to billet definition and costs, 
task definition, system reliability, task training, or training devices). 

Other life-cycle cost elements such as spares, support and test equipment, and 
technical documentation are not addressed. 

One paper summarized in Section II was the HARDMAN Life-Cycle Cost Model 
(HLCCM) developed for the Navy (8). This model was specifically designed for 
performing a wide range of system tradeoffs. The complete program consists of 
several interactive life-cycle cost models that allow the designer to change 
input values for one or more of the life-cycle cost variables and observe the 
impact on the relative cost of the other support elements. The cost variables 
include (1) production, (2) wages, (3) spares, (4) support and test equipment, 
(5) maintenance, (6) technical documentation, and (7) maintenance and operator 
training. For example, if a new, expensive acquisition demands a large number 
of highly skilled maintenance people, it may be worthwhile to examine the 
impact of reducing the tasks and skill level by investing more money in 
support and test equipment to prevent the reliability from decreasing and 
subsequent spares from increasing. Similar to other life-cycle cost models, 
the HLCCM technique seems to explore more adequately the range of possible 
cost tradeoffs. 


7. Summary of Papers Encompassing the Complete HCM 

One immediate implication of the preceding discussion is that 
there are apparently very few modeling processes that cover the complete MPT 
arena. Indeed, the HCM seems to be the one process that achieves this level 
of coverage. However, another MPT procedure examined, which appeared to be 


5-11 



designed to the same level of detail, was the Air Force "Acquisition of 
Supportable Systems Evaluation Acquisition Technology (ASSET)" methodology 
(27). One of the reasons the Air Force procedure demonstrates reasonable 
equivalence to the overall HCM is because ASSET was originally designed by the 
same contractor. In its present form, ASSET represents the combined past and 
present design efforts of the HCM design team, Westinghouse Corporation, and 
the Air Force (27). 

In contrast to the HCMs six major steps, ASSET has eight: 

(1) Define the specific application and identify major data sources 
and planning documents. 

(2) Construct the consolidated data base (similar to the HCM CDB). 

(3) Define all operator and maintenance tasks, tools, and test or 
support equipment. 

(4) Develop maintenance action networks. 

(5) Establish other logistic support resources based on the task 
analysis and maintenance network. 

(6) Compare the new system alternative designs against a baseline 
system to identify different maintenance and resource demand 
scenarios . 

(7) Perform a life-cycle cost assessment using all of the above 
logistics support inputs. 

(8) Develop a design option decision tree that indicates various 
tradeoff situations critical to logistics and human resources. 

Although the Air Force technique seems to use more detailed modeling 
routines, it admits to having some data availability, data manipulation, and 
subroutine integration problems (e.g., problems with obtaining data for 
maintenance network probabilities or the mismatch between the ASSET personnel 
availability model and the equivalent information generated by the Air Force 
Manpower and Personnel Center) (27). 

In summary, it seems that the HCM does not strongly diverge from either 
standard DOD MPT doctrines or other accepted MPT modeling practices. Although 
distinctions were made throughout the preceding discussion, the following 
section summarizes the major positive and negative differences more concisely. 


E. COMPARISON OF DOD AND OTHER MPT GUIDELINES AGAINST HCM 

The major differences between the HCM and the other various MPT modeling 
techniques were pointed out in the preceding paragraphs. In some cases the 
distinctions of other modeling practices represent improvements because they 
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address real-world considerations that could affect the projection accuracy. 

In other situations, the HCM strengths provide a stronger theoretical founda- 
tion. Table 5-1 summarizes the areas in which the strengths and weaknesses 
were apparent . 

Table 5-1 clearly indicates that overall the HCM is fairly similar to 
other MPT models in its logic, use of associated MPT parameters, and input 
data. The major differences considered as weaknesses originated from (1) using 
a rectangular manpower function, (2) not making allowances in the manpower 
flows for the possible impacts of major socio-economic fluctuations, (3) not 
considering long-range manpower trends on personnel availability, and (4) not 
considering all operations and support variables in the tradeoff and life- 
cycle cost assessments. Item (1) above is important because of the potential 
life-cycle end point problems cited earlier during the discussion of Putnam's 
approach. Enke's point reiterated in item (2) cannot be overlooked because 
some socio-economic impacts, such as a rapid defense drop-off with change in 
administration, are not minor perturbations. Perturbations such as this can 
affect the supply of manpower for several years. Item (3) is an important 
consideration because the normal seven- to ten-year acquisition period far 
exceeds the two-year manpower planning projection used in the Army and HCM. 
Admittedly, the Grissmer, Lindsay, and Barnes papers referred to in Table 5-1 
do not include a confidence factor in their manpower projections. However, 
the use of confidence intervals is standard practice and could easily be 
incorporated. Recommendations such as this are discussed in greater detail 
later in this report. The last area, item (4), was considered a weakness 
because all life-cycle elements must be considered to completely assess costs 
and design tradeoffs. 

The HCM clearly exceeded some of the other studies in the areas related 
to building the data base and describing tasks and the personnel pipeline 
calculations. The data base and task area descriptions were comparatively 
better than the Fong or Gordon papers because these studies were not 
projection models. The advantage of the HCM over the series of papers related 
to the pipeline projection was the optimization routine. By not requiring 
either manpower ceiling or starting pool data inputs, the model reduced the 
chance of contaminating the outputs with incorrect or obsolete data. 

Having identified some of the strengths and weaknesses of the HCM via 
the literature search, the next step was to investigate some actual applica- 
tions of the methodology, drawing on the familiarization provided by the 
preceding comparative examination. Inferences about the overall face validity 
of the HCM were withheld until the audits were completed, and any obvious 
parallels between the literature search findings and applications analysis 
were identified. The final results are provided in the following paragraphs. 

F. USE OF AUDITS TO EXAMINE FACE VALIDITY 

The audit phase of the face-validity analysis was designed to follow the 
literature searches so that the inquiry might be better organized (having 
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Table 5-1. Comparative Overview of MPT Methodologies 
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Step 6 Conduct Assessment Group, X X 

tradeoffs ASSET (Assessment Group) (ASSET) 



obtained some foreknowledge of the HCM process and its shortcomings). As 
stated earlier in this section, the three applications selected for the audits 
were SINCGARS, RPV, and DSWS. These three systems were chosen primarily 
because they represented the most recent applications of the HCM. These cases 
were also selected because, being at different stages in the acquisition 
process, they also typified different levels of study detail that is useful 
for examining potential data availability or credibility issues. Finally, the 
RPV analysis was executed by a different contractor team than the one 
responsible for SINCGARS and DSWS. 

As an introduction, SINCGARS is a sophisticated inter-communication 
radio system consisting of a manpack subsystem (i.e., radio backpack) and six 
other radio configurations that can be mounted on different types of vehicles, 
i.e., trucks, tanks, etc. (28). The RPV, a small, drone-type aircraft that is 
remotely piloted from a mobile command center, provides surveillance informa- 
tion on enemy target locations (29). Last, the DSWS system is a self-propelled 
Howitzer linked with three other respective ammunition resupply, operations 
center, and maintenance vehicles (30). 

Each design is fairly complex because several major components make up 
the complete system. Given the time and budget constraints for the total 
evaluation, it was decided that only select components from each system could 
be examined in any depth. The three following selection criteria were 
developed in an effort to choose key components that could surface validity 
problems : 

(1) The component should be a major manpower driver in terms of 
quantity and skills. 

(2) The component should require maximum exercise of the mechanics and 
judgmental portions of the methodology to explore validity issues 
such as error sources, data availability, logic, and credibility. 

(3) Where possible, the component should demonstrate the least 
quantity and quality of source data, therefore requiring an 
exercise of the most judgmental portions of the methodology. 

These criteria would provide insight into the most likely areas 
where error sources, credibility, logic, or ambiguity problems 
could arise. 

Although it was difficult to find components that simultaneously met all 
criteria, generally the four to six components selected for each system 
satisfied at least two of the guidelines. Once the studies were reviewed and 
the components analyzed, a detailed list was assembled that addressed all the 
areas where various face-validity issues could arise. The next step was to 
interface with the HCM designers to trace and understand exactly what was done 
to confirm the problem areas. This was accomplished by converting the problem 
list into a questionnaire, visiting the contractor's facility, and replicating 
in detail the analysis performed (31). The complete questionnaire is shown in 
Appendix C. 
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Two caveats pertaining to the audits should be stated at the onset. 
First, only the RPV and DSWS audits were actually conducted at the con- 
tractor's facility. The SINCGARS audit was performed off-site under the same 
guidelines as a test of the internal reliability of the methodology (32). The 
test of internal reliability requires that an independent group of trained 
individuals, given the same data inputs, attempt to replicate the results of a 
previous application of the methodology. Although the details of the internal 
validity test are provided later in Section VI, it suffices to state here that 
the face-validity issues surfaced during the on-site RPV and DSWS reviews also 
appeared in the separate SINCGARS replication. 

The second caveat applies to the presentation and interpretation of the 
audit results. Appendix C contains approximately eighty questions related to 
both the three specific systems and the methodology as a whole. It should be 
noted that of the eighty issues examined, which covered the complete 
methodology, roughly 85% (or 68 of the potential issues) were adequately 
addressed and resolved during the audits. A summary of the remaining problem 
areas follows . 


1. Results of the SINCGARS Audit 

The major findings of the SINCGARS audit revolved around the 
clarity of the methodology, usability of the technique, and data management. 
The clarity issues primarily applied to the early Step 1 and 2 analysis done 
in support of the ultimate manpower and personnel projections. For example, 
functional requirements had to be developed as a subset of selecting 
predecessor components to both construct a reference system and load the 
historical data bank. The participants in the replication found that the 
detail to which the requirements could be defined had a direct relationship to 
the number of technically different components selected to build a reference 
system and, therefore, could affect the number of people needed to support the 
system. There were no clear guidelines provided for determining the depth to 
which the requirements should be developed. 

Another area of contention developed over the construction of the 
reference system. The reference system is constructed using predecessor 
components that meet the functional requirements and, in turn, are similar to 
the proposed system(s) one desires to model. The study group discovered 
through the SINCGARS example the possibility that, if two or more contractor- 
proposed designs differed substantially, it could be very difficult to con- 
struct one reference system similar to both designs. This dilemma could 
result in a weaker manpower projection for one case than the other. Although 
the obvious solution was to develop two reference systems, the larger problem 
was the matter of defining how much two proposed designs could diverge before 
one reference system was insufficient to resemble the performance of both. 

One area closely associated with the manpower projection is the process 
of evaluating design differences between the predecessor and proposed com- 
ponents and defining the fractional improvements in maintainability and 
reliability performance. The first major difference occurred over the selec- 
tion of the so-called "perturbation values," which are the ratios used to 
adjust reference data to better represent the performance of the proposed 
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component. The study participants found that, because there were not estab- 
lished guidelines for selecting perturbation values, their perceptions of 
certain design improvements varied widely. This variance could have a major 
affect on the manpower outcomes. 

The study group suffered similar confusion over guidelines associated 
with the selection of indirect productivity and induced failure values. 

Although distinct values were provided for SINCGARS, the group recognized that 
the MACRIT, which is the source of these values, provides a range of selec- 
tion. Again, not being given guidelines in the methodology on how to select 
either a high or low factor over a fairly wide range of potential values, the 
study participants felt that the final manpower projection could be affected 
considerably. 

The last, most important, difference was associated with a misunder- 
standing of how the perturbation values were actually used in the manpower 
calculation. It was discovered that the HCM presently uses perturbation 
values only to fill data gaps in the proposed designs. Occasionally, HCM 
analysts find that contractors do not provide performance data on all the 
components in the proposed design (such as SINCGARS). While constructing the 
reference system out of similar predecessor components, the analysts will 
select the predecessor component equivalent to the missing component in the 
proposed design and "perturb" the historical performance data in order to 
complete the data set for the new design. However, in the final manpower and 
personnel projections the total reference system predecessor data are not 
upgraded (using perturbation values) to allow the complete new design to be 
modeled based on proven historical information. This procedure represented a 
major logic problem to the control group because one of the primary reasons 
for modeling new system manpower requirements is to prevent contractors from 
being overly optimistic about the impacts of new technology improvements. By 
not upgrading the old technology to a roughly equivalent proposed configuration 
and establishing a revised historical data base against which to equally com- 
pare proposed designs, the methodology cannot test the integrity of con- 
tractors. Without making an equivalent technology-based comparison, the same 
trivial outcome will always result; the new technology will appear to perform 
better than old technology (i.e., the new technology will require either fewer 
or lower-quality skilled people). 

One area of contention that arose over the personnel assessment portion 
of Step 2 in the methodology was the matching process between tasks and 
skills. The participants found that the cookbook approach outlined in the 
methodology was not completely accurate. This definitive approach is normally 
followed up with the analysts actually contacting various Army training 
schools to confirm, or update, the task and skill matchups. Because this 
caveat was not passed on to the participants, there was general disagreement 
between the skills selected by the original methodology designers and the 
skills selected by the control group. 

The last two areas pertaining to the usability and data issues were more 
related to the methodology as a whole. The analysts discovered that, although 
they jointly had backgrounds in mathematics, engineering, and personnel, they 
still lacked the range of training and experience necessary to completely 
comprehend the pitfalls and nuances of the methodology. The overall con- 
clusion was that the methodology could not be properly exercised unless a 
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well-seasoned, multidisciplinary team having joint military, engineering, data 
management, and personnel backgrounds was employed. The last problem area, 
data management, was not necessarily viewed by the study participants as a 
shortcoming of the methodology. The observation was made that the complex 
nature of the MPT issues alone caused the methodology to contain many places 
where errors could be introduced due to (1) the prodigious quantity of data 
required and (2) the large amount of data manipulation required. 


2. Results of the RPV Audit 

The RPV audit surfaced several potential validity problems 
involving elements of the early data-sorting process, manpower and training 
projections, as well as the later tradeoff analyses. The most important 
finding related to the initial data base step was the lack of a clear decision 
process for selecting and using different sources of performance data (e.g., 
reliability and maintenance data). For example, it was stated earlier in this 
section that the methodology normally requires historical data to be used for 
modeling new system performance. However, in the RPV case, most of the 
components had already been prototyped so that actual reliability and 
maintenance test data existed. Nevertheless, the decision was made to use 
only some of the contractor test data. The remaining maintenance data were 
discarded and relaced by comparable information listed in the design speci- 
fications that were part of the original proposal. The reason cited for this 
action was that experience dictated the contractor data was not reasonable. 
Another example of non-distinct direction for data selection pertained to 
modeling component performance on the small RPV using comparable components on 
full-sized aircraft. The reason the contractor opted to use this information 
was because the Army, having no history of using drones, did not have com- 
parable predecessor data. Closely associated with this decision was the fact 
that the original designers did not differentiate between the different 
performance and environmental envelopes of the aircraft. These differences 
could affect the stress levels and resultant wearout rates experienced by the 
RPV components. The confusion over the selection of data sources, although 
somewhat linked to concerns about accuracy, more specifically revolved around 
(1) identifying the various data sources and ranking them as a function of 
quality and (2) determining when to use one source in preference to another. 

In the manpower and personnel step following Step 1, the analysts found 
the same problems with selecting perturbation and induced failure values that 
were discovered with SINCGARS. Similarly, the step of matching skills with 
tasks lacked a clear audit trail as to how the final skills were selected. 

This particular audit problem was derived from the fact that the Army did not 
list skills for operating and maintaining drone-type aircraft. Therefore, the 
HCM analysts had to search outside the normal cookbook procedure to define the 
new skill areas and obtain associated wage and training information. Regarding 
the final manpower projection made in Step 2 of the methodology, the step 
function deployment assumption discussed earlier in the literature search was 
confirmed. As indicated before, this assumption is considered a problem 
because of the potential personnel allocation conflicts imposed by other 
acquisitions under concurrent development. 
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Step 3 of the methodology, the training resource assessment, surfaced 
some interesting problems with respect to RPV. The analysts discovered that, 
although the methodology provides for a very detailed instructor, training 
resource and cost asessment, it has only been marginally exercised because of 
lack of data. Critical training considerations such as the identification of 
safety related tasks or task difficulty were not highlighted. Additionally, 
the audit team found that the methodology did not differentiate between the 
instructor contact hours required for critical versus non-critical tasks. The 
analysts felt that this was particularly important because RPV appeared to be 
a new technology area not previously employed by the Army. 

An in-depth investigation of the personnel shortage projection in Step 5 
of the RPV study was very revealing. The investigators found that, although 
the modeling algorithm for calculating shortages was sound, the weakness 
pertaining to the use of the short-term manpower availability projection cited 
in the previous literature summary was verified. 

In the final step of the RPV study, tradeoff analysis and conclusions, 
the analysts experienced considerable confusion over the criteria for 
(1) performing certain tradeoffs and (2) identifying the one MPT projection 
that represented the best estimate of the real-world performance of the 
proposed system. For example, in the training assessment several cases were 
found where the design dictated the use of low pay grades in certain task 
areas. However, when the training courses were structured based on pre- 
decessor systems, it was found that rather high skills were actually required 
for the same tasks. Although the study indicated that this mismatch was a 
major conflict, the training and cost implications were never pursued in the 
tradeoff analysis. This ambiguity about the relative importance of various 
tradeoffs did not give the analysts confidence that (1) all possible tradeoffs 
had been considered; (2) important tradeoffs, such as the above example, had 
not been overlooked; and (3) the tradeoffs finally pursued represented the 
pivotal issues. The largest source of ambiguity in the RPV study revolved 
around the presentation of the final MPT and cost projections. The present 
summary format for the projections displays both the reference and proposed 
results together. However, where wide variations between the projections 
occurred (such as SINCGARS) or when several proposed systems were presented 
(such as RPV), it was not clear which set of results represented the most 
accurate approximation and MPT planning guide. The audit team found that the 
source of the confusion originated from not having an indication of the 
relative credibility of the data inputs and outcomes. 

One final interesting aspect noted by the analysts applied specifically 
to RPV but not to SINCGARS. The observation was made that because the RPV 
study was completed roughly one year before this evaluation, the members of 
the original HCM study team did experience memory decay with respect to their 
decisions and judgments about the use of certain data sources and the ultimate 
selection of one of the proposed system designs as the best planning figure. 
This last finding was important because it demonstrated that, even with a 
seasoned corporate bank of knowledge, the judgment areas in the methodology 
must be better structured to allow a firmer audit trail. 
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3. 


Results of the DSWS Audit 


The findings of the DSWS audit were quite similar to both SINCGARS 
and RPV. The main reasons for the similarities were (1) the DSWS reference 
system vas compared against several, widely varying proposed designs; (2) a 
significant amount of data sorting and selection was required; and (3) the 
original DSWS analysis was also completed well before the methodology validity 
evaluation. The chief problems encountered by the investigators related to 
item (1) above were those associated with defining the proper level of detail 
for the functional requirements and building one reference system to model 
four widely varying contractor concepts. Like the SINCGARS audit, it was felt 
that the lack of sufficient detail on functional requirements and singular 
reference system constraints could cause major variance and errors in the 
operational and maintenance manpower support projections. Item (2) above was 
extremely important because of the confusion generated in trying to sort and 
categorize generic component repair times by specific task, establish the 
credibility of the various data sources, and understand which data sources had 
priority over others. The DSWS data question was analogous to the earlier 
defined RPV data problem. Item (3) addresses the audit trail decay dilemma 
similarly revealed with the RPV study. 

As was the case with both the SINCGARS an RPV investigations, the 
analysts did not clearly see the selection criteria for the DSWS perturbation 
values, indirect productivity factors, and final choice of skills. Finally, 
the investigators were confused by the way the data were presented for the 
final design option selected as the MPT planning guide. Similar to RPV and 
SINCGARS, the study team felt that the best option should have been 
respectively chosen as a function of (1) the relative quality of the various 
data inputs and (2) a comparison against a reference system that had been 
perturbed so that the historical and proposed designs were compared on an 
equal technological basis. 


G. VALIDITY IMPLICATIONS 

The preceding discussion highlighted many strengths and weaknesses of 
the HCM. The overriding strengths were: 

(1) The methodology conformed well with DOD and Army doctrines. 

(2) The technique basically resembled other known, accepted MPT 
modeling techniques in its logic, use of MPT variables, and input 
data. 

(3) The methodology was reasonably well supported by its data sources. 

(4) The HCM surpassed most other models in its completeness and 
integration of various MPT modeling techniques . 

(5) The methodology provided a better modeling foundation in some 
areas than other simulations (e.g., manpower pipeline projections). 

(6) The HCM stood up reasonably well under an extensive audit (i.e., 

85 percent of the test issues were adequately answered). 
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However, the audits confirmed some problem areas outlined in the 
literature search. During the process of replicating the three case studies 
the remaining 15 percent of the test issues revealed that (1) the HCM 
conformed with the MACRIT in its use of a rectangular manpower curve with no 
consideration of the Norden-Raleigh reduced end point demands; (2) the HCM did 
not address Enke's potential impacts of major socio-economic fluctuations on 
personnel flows in and out of various skills, and the Army as a whole; (3) the 
technique did not project manpower availability and shortages at the actual 
projected deployment time of the new system; (4) the HCM incorporated, but did 
not exercise, a technique similar to Winston's subjective critical task 
assessment; and (5) the HCM MPT system examined only a small number of life- 
cycle cost tradeoffs (e.g., wages, reliability changes, task redefinition, 
training programs and devices). Additionally, in the data base construction 
phase, early in Step 1 of the methodology, there was a wide variance in (1) 
the types and sources of data selected, (2) the reliability and maturity of 
the data, and (3) the methods used in screening and manipulating the data to 
conform to the methodology or to fill data gaps. This wide variance caused a 
major problem in establishing the credibility of both the analysis and final 
projections. It was also discovered in Step 1 that, by not reaching the 
proper level of indenture for the functional requirements, key tasks and 
subsequent manpower support could be left out. Another major finding 
associated with the reference system portion of Step 1 was particularly 
apparent in the SINCGARS and DSWS audits. It was discovered that where 
several widely varying new designs were being considered, the use of a single- 
reference system could be an inadequate performance comparator for some of the 
proposed concepts. Also linked to the reference system procedure was the 
problem of not adjusting the predecessor historical data as a function of the 
proposed design improvements. This was a pivotal discovery because the 
contractor-projected manpower and costs could not be assessed on an equal 
basis with what would be considered a reasonably equivalent extension of 
existing technology. 

The manpower and personnel projection phase of the audits uniformly 
revealed that a better logic structure was required for selecting perturbation 
values, indirect productivity factors, and induced failure rates. It was 
found that arbitrary selection of these different values would affect the 
ultimate manpower projection, particularly where small variations in manpower 
support (6uch as at the field level) could cause critical shortages. In the 
personnel selection step it was found that HCM's actual skill matching process 
extended beyond the scope of the stated methodology. The resultant skill 
mismatch and cost impacts were felt to be potentially sizeable. 

In the tradeoff and conclusion portion of the methodology, Step 6, the 
audits revealed a definite need for (1) a clear structure for selecting 
tradeoffs and (2) a subjective weighting mechanism of the various data and 
judgmental elements of the methodology to establish the credibility of both 
the final design selected and the accuracy of the analysis as a whole. 

Overall, the findings of the audits suggested that the structure of the 
judgmental portions of the methodology still needed to be improved to offset 
the observed audit trail decay. Furthermore, the sophistication of both the 
MPT problem and the HCM design did not make the process amenable to use by lay 
people. Although the preceding discussion indicates the various sources of 
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potential errors, logic problems, ambiguities, or credibility issues, Table 5-2 
more concisely illustrates the link between these findings and complete array 
of respective face-validity elements defined earlier in this section. 

The face-validity analysis provided an indication of the strengths and 
weaknesses of the HCM, as well as a first-cut, general examination of the 
integrity of the technique. The next step was to investigate and tailor 
validity tests to examine more specifically the accuracy and reliability of 
both the overall methodology and its projections. The validity tests selected 
for the reliability portion of the study, selected to conform with the project 
time and budget constraints discussed in Section III-E, are discussed in 
detail in Section VI along with the results of the complete reliability 
analysis . 
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Table 5-2. Summary of Audit Results in Terms of Face-Validity Elements 
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Face Validity Elements 


• 

CM 

I 

m 

CU 

i— • 

X* 

CO 

H 


5 ^ 


H 

4J 


CO 

•H 

g 

•H 


CU 

4J 

•r4 

H 

c 


xi 

CU 

•H 

0 

4-1 

*0 

u 

o 

0) 

p* 

Pm 

u 



o 



CO 



4J 

as 

CO 

H 

s 

CO 

3 

o 

CU 

cn 

3 

o 

cu 

60 

o 

pc: 

• H 

u 


x> 

Pm 

p. 

e 


o 

<2 



<U 






X* 

CO 

AJ 

CO 

4J 

o 

H 

CO 

55 

• H 

O 


CO 



> 



< 

cu 



H 


CU 


V4 

o 

•H 

O 

vm 

CO 

V4 

3 

CO 

»M 

o 

o 

w 

CO 

Pm 






<D 

»— ( 


3 

•H 


c r 

CO 

73 

•H 

CO 

<U 

c 

w 

CO 

-JC2 


P 

O 

4J 


CU 

o 


H 

55 



4J 


0) 

U 


fH 

<u 

o 

x> 


• H 

•rM 

Vm 

60 

CO 

o 

O 

<0 

o 

XI 

O 

c 


Pm 

M 



X 


X 


co 

o 

o 




CO 


CU 




4-1 

a 


rd 

u 

CO 

CO 

o 

CU 


o 

o 

4M 

CO 





4M 

CM 

0) 

C 

CO 


O 


o 

U 

o 



1 

CO 

CU 

< 

•H 



<u 

•H 

73 


4-» 

PC 


4M 

U 

CO 

a 

o 

o 


•r4 

<U 

u 

CD 

c 

rH 


rH 

4-> 

4J 

H 

3 

o 



•H 


X3 


73 


CU 


PO 

o 


O 


4J 

u 

c 

X 

CO 

4= 


a) co 


•H 

PL, 

CO 

4J 


H »H 


u 



0) 

\C 

a. co 

co 

U 



s 


6 5^ 

CU 

3 




a 

o »-h 

rH 

73 




aj 

a co 

U 

3 




4J 

c 3 

c 

O 




CO 

M CO 

p 

O 



CO 




rH 

•U 


•H 


4M 


<v 

o 


CO 


O 


& 

3 

M 

>> 





7) 

O 

H 



as 

C 

4M 

CO 


CO 

•H 


o 


3 


O 

CO 

V 

o 

CO 

CO 


CU 

P 

> 

a 

u 



73 

4J 

as 

o 

73 




JC 

1 co 

4J 

c 


73 

4J 


•H CU 

co 

CO 


a) 

•H 

o 

4J 4-» 

o 



> 

73 


H 

•H 



u 

3 


i 

73 

MJ >N 


0) 

co 

•U 

3 

•H 4J 


CO 


e 

CO 

•H 

H »rM 



u 

V 

^ c 


CO i-4 


O 

4J 

g 

'O *H 

4M 

3 

rH 


CO 

<u 

<U i— » 

O 

a* P 

H 


U 

u 

c a 


•r4 

CO 

.3 

o 

•l-l 

O *r4 

X 

CO 73 

u 

PO 

a 

3 

CO O 

o 

4J (U 

0) 

•H 

u 

cr 

CO co 

CO 

CO U 

> 

r-M 

o 

0) 


p 

73 a 

o 

CO 

u 

OS 

CO 73 


5-25 


the methodology 



SECTION VI 


RELIABILITY ANALYSIS 


A. OVERVIEW 

As explained in Section III, the reliability, or accuracy, of the HCM 
was explored via three avenues. The reliability of the HARDMAN methodology 
was assessed by examining internal reliability, to a limited extent an 
examination of the experience of individuals who have already applied the HCM 
(or other similar methodologies) to real-life applications, and event series 
validity . 

For the internal reliability test, it was decided to use one of the Army 
test cases (i.e., SINCGARS). A controlled experiment was designed to 
establish whether a group of individuals, trained to use the methodology, 
could both replicate the original answers and also demonstrate a low degree of 
variance in their answers. A group of seven individuals (four civilians and 
three career Army soldiers) of varying backgrounds, were selected for the 
experiment. Given the small sample size (primarily due to restrictions in 
available people, time, and budget) it was decided to dispense with detailed 
statistical comparisons and simply establish general conclusions about the 
repeatability and consistency of the results. The replication was also used 
to develop a list of critical factors that have major impacts on MPT 
projections. For purposes of consolidating the internal reliability findings 
and to find ways to improve future versions of the HARDMAN methodology, each 
participant went through a formal debriefing. Furthermore, a discrepancies 
resolution conference, attended by the study group, ARI, and HCM designers, 
was held following the completion of the replication. 

Another important step in the reliability examination was a qualitative 
accuracy check of past Air Force and Navy experience with the same (or 
reasonably close) methodology. This was done by personally contacting 
pertinent project personnel. 

The last step in the reliability examination was a manpower sensitivity 
analysis. Sensitivity analysis is a useful tool for testing how well a 
technique like the HCM must conform with the real world. This type of test is 
often referred to as an "event series validity test" because it ultimately 
provides an acceptable accuracy or reliability envelope for the outcomes of a 
modeling process. If the envelope is rather large, then a quick-and-dirty 
approximation is acceptable. As the envelope shrinks, greater emphasis must 
be placed on improving the quality and accuracy of each data input and 
algorithm. In this evaluation, the sensitivity (or event series) analysis was 
done last to assist in formulating and prioritizing the closing recommenda- 
tions for improving potential problem areas in the methodology. To obtain an 
indication of the required accuracy of the HCM, a simple top-down cost 
sensitivity analysis was designed to establish the allowable variance in the 
manpower projections which would still provide accurate MPT estimates. The 
estimates of the allowable variance in the manpower projections were performed 
for one sample acquisition using an actual Army system. Each element of the 
reliability analysis is described in greater detail in the next subsection. 
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B. 


APPROACH 


1. Internal Reliability 

The replication effort for evaluating internal reliability 
involved the application of the first two steps of the HARDMAN methodology to 
the Army's Single Channel Ground Airborne Radio System (SINCGARS). These two 
steps were: 

(1) Establishing a consolidated data base. 

(2) Determining manpower requirements . 

The remaining four steps were not included for several reasons. First, 
preliminary review of the HCM process suggested that there were a substantial 
number of judgments, having considerable influence on the manpower estimates, 
in the first two steps. Second, at the time the replication was performed, 

HCM designers had only completed their analysis on the first two sections of 
the methodology. Third, other duty commitments mandated that the experiment 
be simplified so that the Army participants could finish the replication 
within a maximum time frame of three weeks. The SINCGARS components included 
in the replication were as follows: 

(1) Receiver/Transmitter Unit (R/T). 

(2) Electronic Counter-Countermeasures Unit (ECCM). 

(3) ECCM fill device. 

(4) Securable Remote Control Unit (SRCU). 

These components were confirmed as offering the greatest number of judgment 
exercises . 

A group of seven individuals with varying technical backgrounds (i.e., 
math, engineering, psychology) and experience (JPL, Soldier Support Center, 
Army Research Institute) were selected for the testing. This group attended 
the HCM training course, which was held at the HCM designer's facility in 
Boston, Massachusetts, during February 1983. The goal of the course was to 
familiarize the participants with the key analytical and judgment areas in the 
first two steps of HARDMAN. During the training sessions, the methodology was 
applied to an example subsystem of the Army's Multiple Launch Rocket System. 

For the SINCGARS replication effort the two steps were divided into 
eight different data packages. Each package required the determination of key 
judgment and calculational outputs and was preceded by (1) a problem statement 
related to the judgments that were required, (2) a list of the enclosed data 
items to facilitate making the judgments and necessary calculations, and (3) a 
brief statement about how to use the data. The problem statements and descrip- 
tions of the data packages are given in Table 6-1. 

In general, the participants were to proceed independently and 
sequentially through the eight data packages until the four components of 
SINCGARS identified for the replication effort were evaluated. Three weeks 
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Table 6-1. Data Package Problem Statements and Descriptions 


Problem Statement 

Date Package Description 

Develop system functional 
requirements 

Descriptive data for proposed system 
and one worksheet for each subcomponent 

Construct reference system 

Candidate list of predecssor systems 
and description of contractor proposed 
systems 

Define tasks 

List showing range of generic classes of 
tasks and! contractor task definitions 
for proposed systems 

Reliability and maintenance 
determinations for inherent 
versus induced component 
failures and assignment of 
task echelons 

Task time data, component failure data, 
induced failure data, equipment break- 
down structures, predecessor task echelon 
assignments - one worksheet for each 
component 

Determine design differences 
(design difference analysis, 
impact of design differences, 
derivation and application of 
perturbation values (PV) to 
appropriate tasks) 

Data extracted from 1, 2, and 4, above, 
summaries of contractor correspondence 
with component contractors and one 
worksheet for each component 

Select and assign MOS 

List of candidate MOS from which to 
choose appropriate specialties for each 
task 

System/ component 
usage rates 

Select data extracted from the operational 
and organizational plan 

Determine workload and 
manpower requirements 

Data extracted from 1-7, above, available 
manhours - one worksheet for summary of 
results 


were allowed to complete the SINCGARS replication. The study group was 
permitted to ask the HCM designers questions of clarification only by way of a 
mediator. It was originally planned to provide feedback to the students at 
selected audit points in the analysis for the first two SINCGARS components 
(to allow mid-course corrections) and to provide no feedback for the last two 
components to get a feel for what the end-point variance might actually be, 
given no student coaching. In actuality, because of the huge quantity of data 
involved and the relatively short time frame, feedback was given to the 
students after each data package completion for all four components. 
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The internal reliability was primarily assessed by reviewing the study 
group' s answers to the data packages and by comparing them to the baseline HCM 
designer answers. Besides a reliability measure, convergence of answers would 
suggest that the judgment activities could be done by any individual with 
appropriate training whereas divergence would indicate inadequacies with the 
methodology, student background or experience, or HCM training (or possible 
combinations of these problems). Another major step in the analysis was a 
structured telephone debriefing of each participant to identify subjectively 
major problem areas in the replication or methodology. 

The analysis of each data package within the reliability effort involved 
either a subjective review of the participants' responses or a numerical 
evaluation of the key answers as compared to those of the HCM design team. To 
protect the confidential nature of the replication, the participants were 
referred to by number only (from 1 to 6). After the replication was 
completed, each participant was debriefed to define individual problem areas 
with the packages and to get feedback for possible future improvements in the 
methodology. The debriefing specifically addressed relevance to present 
assignments, difficult portions of the replication, discrepancies between the 
HCM designer and study group answers, and the adequacy of the HARDMAN training 
course (see Appendix D for a sample debriefing questionnaire). 

The last major component of the internal reliability analysis was a 
discrepancies resolution conference, which was held during August 1982 at the 
Soldier Support Center at Ft. Benjamin Harrison, Indiana. This conference was 
attended by the study group, representatives of JPL, the Army Research 
Institute, and the HCM designers. The purpose of the meeting was to provide a 
structured means of resolving differences and misunderstandings resulting from 
the replication. This conference was a two-day activity in which each of the 
eight data packages was examined and discussed in detail. 


2. Qualitative Accuracy Check 

The qualitative accuracy check was conducted by using participants 
in existing military projects that have used some form of similar manpower 
estimating methodology. The projects covered included the Advanced Lightweight 
Torpedo (ALWT), the DDGX destroyer, the VTXTX flight trainer, and the SUBACTS 
submarine combat system. The pertinent project personnel were asked their 
opinions on the validity of the HARDMAN methodology as compared to other 
methodologies, as well as real-life data. 


3. Manpower Sensitivity Analysis (Event-Series Validity) 

The objective of the sensitivity effort was to roughly approximate 
the permissible variation in the manning variable. To accomplish this objec- 
tive, four tasks were undertaken. The first was to develop a data set of 
actual costs for the SINCGARS system. The second was to develop approximate 
manning data for SINCGARS. The third task was to determine broad, acceptable 
limits of error on total life-cycle cost estimates according to authorities 
engaged in the budget and acquisition review process. The final, and largest, 
task was to determine the extent of error allowable in manpower and training 
cost estimates, given that other cost elements cannot exceed the limits of 
their allowable ranges. 
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Within Task 1, data were collected reflecting typical costs of various 
elements of life-cycle cost. The major components of total cost were 
identified in an attempt to determine approximate ranges. It was decided that 
if possible, the more dif ficult-to-identify elements of life-cycle cost would 
be ignored if their relative sizes were small. In Task 2, actual training, 
manpower, and training estimates associated with SINCGARS were determined. In 
Task 3, a variety of participants in the acquisition process were interviewed 
to determine what guidelines they use for the range of acceptability of 
errors. The groups contacted included OSD or CAIG, who are involved in the 
review of DOD acquisitions. The results of these interviews were resolved 
into perceived admissible limits of error on total life-cycle cost and each of 
its components. In Task 4 the allowable variation in manpower cost in the 
presence of other acceptable errors in the estimate was determined. This 
study used an already developed life-cycle cost and level-of-repair model to 
study the interrelationships between projection errors for several quantities 
(8). The central idea was to determine how large or small the permissable 
error in manning estimates could be as a result of these relationships. 

The first step in Task 4 was to identify all the variables that 
influenced manning and manning estimates. Next, other readiness-critical 
quantities estimated in the model that depended on the same inputs were 
identified. There are five major sources of variance in manning require- 
ments. These are the equipment operating program, the deployment plan, 
reliability, maintainability, and manpower characteristics. Together they 
determine actual manning, and estimates of the variables that describe these 
areas subsequently impact the accuracy of manning estimates. 

The operating program describes the number of hours during which the 
force is expected to operate the equipment. This might be twenty hours a 
week, one or one hundred sixty-eight. During the life of the system, these 
numbers will aggregate to a total usage that will have a fundamental effect on 
total life-cycle cost. But for any given period within the life cycle, the 
rate or pace of operations will help determine the demand rate for repairs. 

The deployment plan simply tells how many units will be purchased and- 
deployed in the field. By extrapolation, the equipment operating hours are 
determined, both over the entire life cycle and per unit of time. The 
deployment plan also usually explains how usage rates are distributed among 
operating units. This is important because of the effect of geographic 
separation on various costs (i.e., pipeline delay times, number of maintenance 
facilities, etc.). For example, a division may generate the requirement for 
only one half a manyear of labor per year to repair a type of equipment. 

While most labor costs would therefore be accounted for at half cost, training 
would not, because a whole man must be trained for each division. 

The reliability of an equipment expresses the rate at which failures 
occur as a result of usage. In company with the operating program and 
deployment plan which determine the usage rate, the reliability measure 
(either mean time between failures (MTBF) or failure rate) gives the demand 
rate for repair services. These three areas jointly determine how many 
failures must be repaired per unit of time. 
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The maintainability of an equipment is expressed by the average time 
required to repair it. In most systems this can be divided into two repair 
times. The first is system restoration time estimated as the mean time to 
remove and replace a faulty component (MTRR). The second is the mean time to 
repair the faulty component after the system has been restored (MTTR). The 
system support policy is usually developed by analyzing the relative costs of 
undertaking these two repair actions in different ways and at different 

places. An item is discarded if the cost of repairing it, including all the 

training, equipment and manpower consumed, exceeds the cost of buying a 
replacement. When the converse is true, then the difference between repair on 
the spot versus repair at a depot must be computed. At the organizational 
level, the system is generally repaired through fault isolation and replace- 
ment of the failed module. The failed module may then either be repaired or 
discarded . 

The last major determinants of manning are characteristics of the 
manpower itself. These will include quality (skills) and a version of 
quantity. The quality of manpower will have an effect on repair times. 

Generally speaking, these skills are set at the lowest level capable of being 

trained through the service schools and cannot be dealt with in the estimation 
process. Quantity is related to the amount of work available from a single 
individual in a period of time. Typical times are 40 to 60 hours a week. 
Depending on the circumstances, these limits may also be exceeded. 

It should be noted that other LCC variables such as special support 
equipment, tooling, etc., can effect manpower and, therefore, represent 
additional manning drivers. However, for the purposes of this top-level 
sensitivity study, variables such as these were not major manpower 
determinants and were therefore not considered. 

SINCGARS was selected as the system to illustrate the error bound 
interrelationships because it had been studied previously in the HARDMAN 
project and because it represented a small (therefore, easy to model) but 
important electronics system. The cost model used to perform the analysis is 
an adaptation of the LCC model summarized earlier in Section V developed by 
the Navy HARDMAN Office (8). The bulk of the modifications made to the LCC 
model were aimed at converting the computer software to a fast turnaround 
system suitable for extensive sensitivity analysis. In addition, certain 
values were added to the printed output, along with the ability to impose 
predetermined support policies at the line replaceable unit (organizational) 
level . 


The model is a level of repair and life-cycle cost model that was 
developed especially to be sensitive to the relationships that drive manpower 
costs. It is roughly based on the MIL STD 1390 group of level of repair 
models although the errors implicit in those models have been corrected, and 
several innovations have been added to the computational procedures (8). 

An example baseline cost estimate is shown in Table 6-2. The SINCGARS 
life-cycle cost estimate is $784 million, excluding batteries. The batteries 
were left out of the cost estimation process because they consumed a great 
deal of computational time and added little to the analysis. Manning (in 
manyears) is summarized in the lower right corner of the table. Real manning 
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Table 6-2. Baseline Life-cycle Cost Sensitivity Analysis Run 


Cost Elements, 

M$ 

System Rates 


Life-Cycle Cost 

783846.0 

Number of LRA Types 

17 

System-Unit Cost 

11.1 

Number of LRAs 

17 

INITIAL COSTS: 


Confidence Against 

0.999 

Production 

312296.0 

Stockout 


Maintenance Training 

1104.2 



Operator Training 

70280.0 

System MTBF 

472.3 

Misc. Manpower 

0.0 

‘System MTTR 

3.0 

Spares 

2512.2 



S+TE 

67048.6 



Technical 

55.2 



Documentation 




TOTAL 

453296.0 

SUPPORT POLICY SUMMARIES: 




LRAs coded COD 

0 

OPERATION AND SUPPORT COSTS: 

LRAs coded MOD 

9 

Maintenance Wage 

27457.5 

LRAs coded local repair 

4 

Operator + Office 

31019.4 

LRAs coded discard 

4 

Wage 




Maintenance Training . 

3390.6 



Operator Training 

215920.0 



Misc. Manpower 

0.0 

TRAINING BILLETS BY TYPE: 


Spares 

9577.7 

Equipment Operators 

28112 

Repair 

1962.7 

Org. and Int. Maint. 

420 

Transportation 

17.6 

Depot Technicians 

6 

S+TE 

41198.5 



Technical 

5.5 

REAL MANNING REQUIREMENTS: 


Documentation 


Org. and Int. Maint. 

205.12 

TOTAL 

330550.0 

Depot Maintenance 

4.27 



OPERATORS 

7028.00 


requirements indicate the total number of manyears of labor, for all skills, 
actually consumed by SINCGARS each year. These manyears are not necessarily 
consumed in full manyear increments because labor requirements are distributed 
over a number of geographical units (28 divisions, in this example). The 
effect of geographical distribution is to expand the real manning requirement 
to a larger number of billets required for training purposes. Because it is 
not possible to train portions of people or to give people part of the 
training required, training billets are constrained to be full integers; $nd 
one must be developed for every individual who is required to work on the 
system, no matter how little time is spent with it. 
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The most extensive manpower requirement is for operators. This shows up 
in the consumption of real manning requirements as well as in the establish- 
ment of training billets. A glance at the list of costs also shows that, with 
the exception of production cost, operator training and billet costs constitute 
the largest elements of life-cycle cost for the SINCGARS example. However, 
while operator costs appear to be important in the estimation of total cost, 
they are unimportant in this study because they are unresponsive to changes in 
most of the input variables that influence maintenance manning. It is 
understood that the relationship between operators and maintainers may vary 
from system to systan. 

The SINCGARS manpack version was modeled as a collection of 17 (18, 
including the battery) line-replaceable units. In fact, the SINCGARS program 
office basically views the equipment as consisting of four modules (R/T, 
handset, whip antenna, battery) among which the R/T breaks down into LRUs. 
Because the other three modules have only one piece and are discard items 
anyway, the equipment's structure was simplified to accommodate the computa- 
tional problem, treating modules as LRUs along with the LRUs that make up the 
R/T. 


The model is driven by inputs that describe the support and deployment 
environment, the system design, and the particulars of component design. The 
data used in this exercise were selected as representative acquisition values 
and are reported in Appendix E, along with the component cost sheets. 

The model operates by estimating life-cycle cost for each component in 
four different support postures. Normally, the least costly of these is 
determined, and the output data associated with it are forwarded to the system 
cost estimate. In the current example, support policies had already been 
determined, and these were used to override the cost-minimizing routine of the 
program. 

At the same time that costs are aggregated by the systan model, certain 
variables of concern are aggregated as well. These include reliability, 
maintainability, and system- con figuration summary data, as well as an 
accounting of the support postures selected for the different LRUs. While it 
is understood that skill tradeoffs cannot be examined using aggregate costs, 
the use of total costs for a top-level sensitivity study such as this was 
acceptable. Finally, the cost tableau is broken down into investment of 
initial cost and operating and support costs. Initial costs include both 
production and logistic investments. Drawing on both the Table 6-2 baseline 
costs and perceived acceptable LCC error bounds, the above cost model was 
iterated until the relationship between manning and the previously identified 
five major LCC variables could be clearly plotted. 


C. SINCGARS REPLICATION RESULTS 

1. Work Package 1 - Develop System Functional Requirements 

The functional requirements data package consisted of three 

subtasks: 


(1) Identification of functions and performance standards of 
SINCGARS Manpack systan. 



(2) Organization of the functions and standards into a 
functional hierarchy. 

(3) Allocation of the functions to equipment and people. 

The HCM designers' answer sheet was a matrix that matched subsystem 
elements and performance standards to functions (see Table 6-3 for a sample 
answer sheet). 

There were no numerical answers to evaluate in this package. Part of 
the study group adjusted the answer format. Additionally, differing semantics 
among the participants complicated the analysis. That is, a "participant" 
could mean the HCM designers but use different wording. Neither of these last 
two factors necessarily made an individual's answers incorrect. In order to 
evaluate the answers, it was decided to determine whether or not the par- 
ticipants included the major functional requirements in their answers. As 
shown in Table 6-4, none of the respondents had the function "Operate 
SINCGARS." However, the remainder of the answers indicated that the study 
group understood that this was included. All respondents had "Communicate" 
and four (2, 3,4, 6) had "Accommodate Mobility," but only two had "Command 
Control." It should be noted that an "X" indicates matching answers between 
the HCM contractor and respondents. 

In terms of format, one participant had three extra element columns 
(adaptor, shorting block, COMSEC), while another individual never used the 
backpack column. Two individuals itemized tasks rather than functions. Three 
individuals missed at least some performance standards. On the other side of 
the spectrum, five of the study group included extra items such as "pass 
through retransmission" and "monitor net," which the original HCM designers 
did not have. In this sense, the study group's answers were more complete. 

In the debriefing it was found that two of the group were confused as to 
the HCM usage of "end item." The study group as a whole was unsure about much 
detail to give in its answers. Regardless of some of the above confusion, the 
major functional areas were basically understood and covered by the 
participants even though they were limited by an open-ended answer format. 


2. Work Package 2 - Reference System Selection 

The objective in this package was to select the reference system 
based on the following given information: 

(1) Functional requirements. 

(2) Reference system candidate equipment. 

(3) Proposed system design data. 

The analysis of this package was based on the ability of the replicants 
to specify the four major system components: R/T, ECCM, ECCM Fill, and SRCU . 

Given the correct selection of the radio/ transmitter (AN/ARC-114), the 
remainder of the components generally followed. As it turned out, all 
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Table 6-3. Functional Requirements Answer Sheet 
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Table 6-4. Work Package 1 - Respondent vs. Dynamics Research 
Corporation Results 


Major 

Functional Requirements 

DRC 

Respondents 
1 2 3 4 5 

6 

Total 

Number 

Correct 

Operate SINCGARS 

X 

Not specified but 





inferred 



Communicate 

X 

X X X X X 

X 

6/6 

Command Control 

X 

X 

X 

2/6 

Accommodate Mobility 

X 

XXX 

X 

4/6 


participants had doubts concerning the HCM designers' answer. Several felt 
that their selections were better suited as the baseline system. In the 
discrepancy review the HCM designers explained that the 114 had the most 
available data. This was obvious from a data source matrix that was not given 
to the study group. This mismatch also suggested that the group did not 
understand the relationship between comparable component selection and data 
availability for that component. 

This package was among the easiest to grade of all the packages (see 
Table 6-5 for the answer comparison). Three individuals (3,5,6) had the 
correct R/T. Two of these three had the remaining major components correct as 
well. As in Package 1, the level of indenture was again a concern. All 
participants were missing at least some detail. Minor subsystems such as fill 
cable and battery were missed by everyone except Participant 1. Two indivi- 
duals had some detail (i.e., programmer gun, antenna) that was not included by 
the HCM contractor. Aside from the doubts on the HCM designer selection, most 
participants felt that the material was too technical, while two participants 
felt that there was not enough time allotted for this task. 

In summary, without knowing the complete HCM contractor rules used for 
the reference system selection, and without all necessary input information, 
it was difficult to obtain the same answer. 


3. Work Package 3 - Task Identification 

The objective of this part was to take the previous reference 
system design as specified by the HCM designers, plus the data included in 
work package 3, and develop generic and reference task lists and assign the 
tasks to equipment. 
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Table 6-5. Work Package 2 - Respondent vs. Dynamics Research 
Corporation Results 


Major 

System Components 

DRC 

Respondents 
1 2 3 4 5 

6 

Number 

Correct 

AN/ARC-114 R/T 

X 


X 


X 

X 

3/6 

SN 416/APX 76 ECCM 

X 


X 


X 


2/6 

KYK-13/TSEC Fill 

X 


X 


X 

X 

3/6 

C-2328/GRA-39 SRCU 

X 

X 

X 

X 

X 

X 

5/6 


The answers to this package were similar to those in Package. 1 in that 
there was a fair amount of subjective matching of tasks to equipment. This 
again complicated the grading. Therefore, it was decided to check for major 
categories of equipment only. A major problem arose for the study group here 
because in Package 3 there were three new major equipment categories (antenna 
coupler, digital data device, COMSEC), which were not included on the original 
work Package 2 answer sheet. All three of these categories were missed by 
almost everyone (see Table 6-6). Only one participant broke his answers into 
generic and reference categories. Even the HCM designers' answers were not 
broken down this way. The HCM contractor did not use the workload worksheets 
for his answers even though they were provided to the participants. Only one 
participant used the appropriate worksheet from the methodology. This had 
been done in the training course but was not a requirement of the replication. 
The level of indenture was again a problem for the study group. Everyone was 
missing at least seme detail. 

The HCM designers pointed out during the discrepancies review that this 
type of effort would typically be a multi-person effort and that missed areas 
could be picked up within later data packages. Almost all of the participants 
felt that either not enough detail was provided or the solution did not 
logically follow from the methodology instructions. Upon clarification of the 
problems within this element of the replication, it appeared that the par- 
ticipants had a reasonable understanding of how to apply tasks to equipment. 


4. Work Package 4 - Reliability and Maintainability (R&M) 
Determination 

In this package the analysts were directed to use given input data 
and then to perform a reliability and maintainability analysis of the SINCGARS 
system manpack configuration. The final result of this effort was the deter- 
mination of the system equipment maintenance workloads. The participants were 
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Table 6-6. Work Package 3 - Respondent vs. Dynamics Research 
Corporation Results 


Major 

Equipment Categories 

DRC 

1 

Respondents 
2 3 4 5 

6 

Number 

Correct 

Man pack 

X 

X 


X 



X 

3/6 

Receiver Transmitter 

X 

X 

X 

X 

X 

X 

X 

6/6 

Antenna Coupler 

X 



X 




1/6 

ECCM Unit 

X 

X 

X 

X 

X 

X 

X 

6/6 

Digital Data Device 

X 







0/6 

SRCU 

X 

X 

X 

X 

X 

X 

X 

6/6 

00 MSEC 

X 







0/6 


to take the tasks and equipment on the workload worksheets, perform the cal- 
culations as specified in the instructions, and ultimately derive maintenance 
ratios (see Table 6-7, column 14, for a sample workload worksheet). This was 
to be done for three types of systems: a reference system and two proposed 
systems, one from Cincinnati Electronics (CE) and one from ITT. This exercise 
resulted in an examination of approximately 300 maintenance ratios for each 
participant. Unfortunately, wrong frequency information was given in the 
instructions for using operating hours. The wrong usage metric (annual 
operating hours) was given for the CE ECCM module. The contractor answer 
sheets had several problems. General and direct support were interchanged for 
the reference system ECCM module. CE SRCU answers were given even though the 
participants were only supposed to use reference system data. A few minor 
components (e.g., battery) were missing. Most importantly, several answer 
sheets were wrong and had to be replaced by corrected ones. 

In view of the above problems, it was decided to grade the study group 
based on what they thought was the proper procedure. The use of reference 
system data for components of the CE and ITT were not included. Because one 
participant (1) did contact the designers and obtain the correct CE ECCM usage 
metric, the use of the incorrectly given value was counted as wrong. Table 6-8 
gives the percentage of correct answers with clarifying comments. The use of 
the correct usage metric for the CE proposed system is obvious for Respondent 
1. His percentage correct was 94 while everyone else varied between 59 and 
71. Several respondents (3,4,5) were missing some system components. Par- 
ticipant 6 disregared some columns on the workload worksheet but still managed 
to obtain high scores. Similarly, Participant 3 seemed to differ from the HCM 
designers in his approach but still arrived at correct answers. 
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Table 6-7 . Corrective Maintenance - Workload Worksheet 
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required, indirect maint. manhours included with MTTR/task manhours because these manhours were derived 



Table 6-8. Work Package 4 - Respondent vs. Dynamics Research 
Corporation Results 
(% Correct-Maintenance Ratios) 


Respondent 

Reference 

CE 

ITT 

Comments 

1 

70 

94 

87 

E 

2 

95 

61 

94 

A 

3 

95 

59 

82 

A, B,D 

4 

40 

• 60 

95 

A, B,E 

5 

50 

71 

89 

A,B 

6 

100 

71 

89 

C 

X 

75 

69 

89 



A - Wrong usage metric for CE 
B - Missing items - Reference, ITT 
C - Missing calculation steps 
D - Different calculation procedure than DRC 
E - Wrong total actions - ITT, Ref 


The study group had several comments on the incorrect and confusing 
replication instructions. Most commented on the extensive calculations, time 
limitations, and tedium caused by the task. One participant (3) resorted to 
using a programmable calculator. 

During the discrepancy review, the HCM designers noted that this 
procedure has now been simplified by using updated LSA data sheets and by 
automating the worksheet. 

Despite all the problems, the number of answers that matched the con- 
tractor's calculations were relatively high (generally between 60 and 95%). 
This indicated that at least for SINCGARS, the study group probably understood 
how to apply this phase of HARDMAN. 


5. Work Package 5 - Design Difference Index (DDI) 

The participants were directed to develop the SINCGARS proposed 
design in the manpack mode for the two contractors (CE and ITT) by evaluating 
and compiling a list of reference system versus proposed system design dif- 
ferences. The study group was then supposed to determine the impacts on 
system workload of each design difference. The resultant perturbation values 
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(PV) were to be used for adjusting the reference system design so that equit- 
able comparisons could be made between the reference and proposed systems. 

This task generated the most discussion during the discrepancy review. 

The discussion centered on the nature of the comparisons. For instance, should 
a reference system be used "as is" or should there be an adjusted reference 
system if reference and proposed systems have significantly different tech- 
nologies and time frames. This question was not addressed during the training 
course or the replication. Because this problem was discussed earlier in 
Section V, it will not be pursued here. 

As in packages 1 and 3, the work that Package 5 answers were quite 
subjective (see Table 6-9 for a sample contractor answer sheet). This package 
had the highest variance among participant answers. The contractor answers 
had only two PVs . Participant 3 had 30 values. Individual 6 had 16. Par- 
ticipants 4 and 5 had none. Participant 5 left out the ECCM, ECCM Fill, and 
SRCU subsystems. Individuals 4, 5, and 6 made no reference to the LSA-02 
reference data. 

It was found in the debriefing that the study group had trouble 
determining the design differences and did not understand the use of the PV. 

In summary, it is apparent from the study group feedback and work package 
answers, that this step could not be closely replicated and further 
clarification is needed. 


6. Work Package 6 - Select and Assign MOS 

The objective in this part of the replication was to record the 
MOS and skill levels that are currently performing each operation and 
maintenance task for the reference system equipment in column 6 of the 
workload worksheet. MOS and skill levels were also to be recorded for the 
contractor proposed systems. Lastly, for those tasks omitted by the 
contractor, the reference system data were to be used. 

Because the study group had previously calculated the column 14 
maintenance ratios on the workload worksheets, it is likely that the 
replication effort would have proceeded more easily if this task had been done 
earlier. This was reiterated during the discrepancy review. Additionally, it 
was pointed out that the MOS selection process needs additional refinement by 
the HCM designers. Presently there are no systematic decision procedures for 
deciding which tasks are to be matched with the appropriate MOS and skill 
levels . 

Once again, there were problems with the given answers. The grader was 
not given all the answer sheets. LSA answers were supplied for components 
where reference data should have been used. Lastly, CE R/T answers were given 
when the study group had not been given the inputs. As opposed to Package 4, 
the reference components jointly included in the proposed systems were 
included in the grading. This inclusion is not very significant (6 out of 151 
answers for the CE (4%) and 13 of 141 for the ITT (9%)), but it was done here 
because the participants were directed to do so in the instructions. 
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All participants primarily had problems with, or could not understand, 
the contractor MOS selection process. In fact, one individual used pre- 
decessor system MOS instead of those of the reference system. Even though it 
was only necessary to copy MOS from the correct data sheets, the average scores 
were not particularly high (55-80%) (Table 6-10). Two possible contributing 
factors are (1) the recurring problems with the methodology instructions and 
(2) the boredom caused by the tedious nature of filling in approximately 300 MOS 
and skill levels. 


7. Data Package 7 - System/Component Usage Rates 

In this package the usage rates for the components of SINCGARS 
were to be determined. This data is required for column 15, usage rate, of 
the workload worksheets (see Table 6-7). Because only one value for each 
component (R/T, ECCM, ECCM Fill, SRCU) was required, this was the most 
straightforward assignment of the entire replication effort. The entire study 
group had the correct answers. Note that in doing a firsthand application in 
the real-weapon development environment, usage data might be difficult to 
acquire. 


Table 6-10. Work Package 6 - Dynamics Research Corporation 
vs. Respondent Answers (% Correct MOS Answers) 



Respondent 

Reference 


CE 

ITT 

Comments 


1 

50 


95 

79 

A 


2 

77 


87 

73 

C 


3 

73 


79 

67 

C 


4 

50 


97 

63 

C,D 


5 

45 


44 

91 

A,C,D 


6 

36 


85 

69 

B,D 


X 

55 


81 

74 


A 

- Missing 

Reference System R/T MOS. 




B 

- Assumed 

DRC used predecessor 

rather 

than 

reference . 


C 

- Missing 

CE or ITT R/T MOS. 





D 

- Missing Reference Components 

for ITT. 
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8. Work Package 8 - Determine Workload and Manpower Requirements 

The goal of this final package was to determine the workload 
results and manpower requirements for the reference and proposed systems. The 
workload results were obtained by multiplying the maintenance ratios from Work 
Package 4 by the scenario usage rate from Package 7. That result was to be 
multiplied by the indirect productivity factor (IPF) to obtain the workload 
results. The next step was to add the workloads, by component, to get the 
total. The manpower requirements were to be obtained by multiplying the 
workload totals by one of two manpower availability factors that were supplied 
by the contractor. Because this last step was trivial, only workload totals 
were analyzed in detail. 

The HCM design team answers had several shortcomings. Calculation 
mistakes were found that had to be corrected, and LSA data were incorrectly 
used in place o^ reference system data. With respect to the study group, the 
use of wrong MOS was evident for five respondents. Reference data were left 
out by four of the participants. Four individuals also made calculation 
mistakes. The participants were confused by the contractor justification for 
selection of IPF and manpower availability factors. Two also commented on 
time limitations. Given the above problems, in addition to the problems with 
the previous packages, it was surprising that the study group got any of 
Package 8 correct. Actually, the average percent correct was between 25 and 
45 (Table 6-11). One individual (2) averaged 69% correct for each of the 
three systems. 


D. QUALITATIVE ACCURACY STUDY 

The names of pertinent military personnel were supplied by ARI and the 
Navy HARDMAN office. These contacts covered a wide range of Navy and Air 
Force procurement projects that required manpower and training forecasts 
(Table 6-12). The feedback on HARDMAN with respect to these real-life 
applications varied from highly positive to negative. It was noted that these 
impressions could be at least somewhat affected by the personal biases of 
people contacted. Nevertheless, as a rough estimate of accuracy, it was felt 
that these contacts were a good initial step. 

The Advanced Lightweight Torpedo (ALWT) project logistician was highly 
favorable (33). He was impressed with HARDMAN in that its initial application 
could start with very little input data. Its resulting projections appeared 
to be within about 10% of actual manpower data. His immediate experience 
applied to the reliability and maintainability areas. He also pointed out 
that these estimates went through the Navy's Logistics Review Council (LRC) 
unscathed. This typically happens only thirty percent of the time. Most of 
those projects that do not pass have unsatisfactory manpower and training 
estimates. HARDMAN also saved ALWT large sums of money because of its ability 
to highlight possible problems. These comments were reiterated by the ALWT 
ILS manager although he added that the HCM was difficult to understand (34). 

He also pointed out that HARDMAN provides a consolidated data base that 
facilitates sensitivity analyses. The ALWT and SIRCS MPT logistic element 
manager felt that once the total integrated methodology (HARDMAN plus training 
requirements determination plus life-cycle costing) is developed, it will be 
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Table 6-11. Work Package 8 - Dynamics Research Corporation 
vs. Respondent Results 
% Workload Results Correct 


Respondent 

Reference 

CE 

ITT 

Comments 

1 

0 

70 

25 

B,C,D 

2 

86 

80 

42 

A 

3 

29 

60 

50 

A,D 

4 

0 

10 

8 

A, B,D 

5 

29 

30 

17 

A,B 

6 

14 

20 

8 

A, B,C 

X 

26 

45 

25 



A - MOS Discrepancies 

B - Component answers wrong 

C - Left out reference maintenance 

D - Did not use all reference data for CE or ITT 


Table 6-12. Qualitative Accuracy Study Contacts 


Project 


Contact 


SUBACTS 

- Submarine Combat System 

Training Manager 

ALWT 

- Advanced Lightweight Torpedo 

ALWT Logistician 

ALWT 

- Advanced Lightweight Torpedo 

ALWT ILS Manager 

DDGX 

- Destroyer 

Personnel and Training 
Analyst 

ALWT 

- Advanced Lightweight Torpedo 

MPT Logistics Element 
Manager 

SIRCS 

- Shipboard Intermediate Range Combat 



System 

MPT Logistics Element 
Manager 

VTXTX 

- Flight Trainer 

Project Manager 

Air Force 

- ASSET and other similar 

ASSET Operations Research 


techniques 

Analyst 
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very useful for the early identification of manpower requirements (35). In 
comparison with other similar methodologies he felt that HARDMAN was less 
cumbersome and more accurate . He also pointed out that one drawback of this 
type of methodology is that it does not interface with hardware design 
requirements . 

The training manager for the SUBACTS program felt that, although the 
HARDMAN manpower forecasts have been fairly accurate, there were problems in 
understanding the approach (36). The SUBACTS program brought in an independent 
contractor to use HARDMAN, which helped make the approach more understandable. 
Concurrently, the new projections proved to be as accurate as the original 
forecasts. 

One of the DDGX destroyer training analysts has had MPT experience on 
pilot programs within NAVC, NAVAIR, and NAVLX. He stated that there was no 
standard, overall methodology for manpower estimates at present although it is 
planned (by January 1984) to evaluate a Navy program (SYSCOM) with the help of 
the HARDMAN office. He felt that, although total feedback on HARDMAN has not 
been received as yet, it is probably quite useful and will have many 
applications within the Navy (37). 

The project manager for the VTXTX flight trainer was too far removed 
from forecasting to say how closely HARDMAN resembled real life. With respect 
to other similar methodologies, he stated that it has typically been difficult 
to match their outputs to actual performance. This has usually been due to 
the differing ways the methodologies define system components as compared to 
the actual system design (38). 

An operations research analyst at the Air Force's Human Resources 
Laboratory has been involved with several Air Force projects. She has worked 
with HARDMAN-type methodologies (e.g., coordinated Human Resources Technology, 
ASSET) but has found to date they have not been widely applied (39). 


E. RESULTS OF MANPOWER SENSITIVITY ANALYSIS 

The problem explored in this portion of the reliability analysis was how 
accurate manning estimates must be in a new system acquisition. Because the 
penalty and cost of estimation can be quite high, it is important to be clear 
about the benefits of accuracy and the costs of inaccuracy, particularly when 
developing a projection technique such as the HCM. Further, because manning 
is not estimated in a vacuum, it is important to establish whether concentra- 
tion on accuracy in other related variables will have any impact on the 
manning estimation. 

In this portion of the analysis, permissable error bounds were 
determined and applied to the SINCGARS program to study the interplay of 
manning estimate accuracy and the accuracy of other variables. Error bounds 
were determined through interviews with decision makers involved in program 
review offices such as the Office of the Secretary of Defense (OSD) or the 
Cost Analysis Investigation Group (CAIG). For this small study, most of the 
individuals contacted were associated with OSD. Data descriptive of the 
SINCGARS system were obtained from the SINCGARS program office. Error 
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relationships were investigated through the use of an interactive life-cycle 
cost model developed for the U. S. Navy and adapted to this particular 
exercise (8). 

The overall conclusion of this brief study is that accuracy of manning 
estimates may not be a significant issue in the development of tools and 
methods for acquisition support and cost analysis. For example, if manning 
estimates were scrutinized by the budget controllers and were found to be as 
poor as the 15% boundaries allow, the error in spare stockage costs (a major 
LCC variable) would be almost 50%. Therefore, budget controllers would most 
likely flag other important quantities long before the manning estimates would 
exceed their permissable limits. This conclusion is explored in greater 
detail in the following paragraphs. 


1. Permissable Error Criteria 

The costs of error vary by circumstance and direction. Many 
believe, for example, that the "cost growth" phenomenon marking system 
acquisitions of the past decades is partly explained by poor cost estimating 
in the early stages of acquisition. Nonetheless, this facet of many programs 
has been a contributing factor to cancellation. 

A theoretician would relate the permissable range of error to the 
trade-off between the cost of error and the cost of reducing it. In the case 
of manning estimation error, the major variables have to do with establishing 
appropriate personnel pipeline flows at the right times to make required 
resources available on time. Conversely, excess flows are to be avoided. 

When these flows are of the wrong size, costs are incurred in adjusting them. 
Costs of concern to the Department of Defense come in two forms: dollars and 
readiness. These are looked upon differently by decision makers. 

OSD (or CAIG) personnel responsible for questioning the estimates of 
cost and support requirements for new systems look for (1) gross error in cost 
estimates and (2) errors in readiness-critical quantities. If total dollars 
is the only issue, an estimated error of about $50 million (in an estimated 
quantity) seems to be required before decision makers are willing to slow down 
the progress of a program to check for more accurate estimates. By the same 
token, very small amounts of dollars may be involved in such decisions if the 
variable being estimated is critical to readiness . Here the rule is that the 
cost estimate may not be off by more than about 15%, regardless of how small 
the total dollar amount is. 

Hie mechanism by which these limits are employed is the application of 
experience and judgment to assess cost projections from program offices. The 
decision makers queried have all studied the relevant issues for many years on 
a variety of systems. It was assumed that their experience provided them with 
a reasonable, independent assessment of the value of specific quantities of 
interest. If their assessments differ from those presented by program offices 
by more than the amounts indicated above, this gives rise to a question and 
request for further analysis. 

To formalize this mechanism, it can be compared to a statistical test of 
hypothesis. The decision maker's estimate is the population mean and the 
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program office submission is the sample mean. The decision maker acts as if a 
15% difference is sufficient to reject the hypothesis that the sample is drawn 
from the same population. 


2. Results of the Analysis 

The procedure followed was to drive manning to +15% of its value 
in the baseline estimate by varying several different input values. The 
results for MTTR, MTRR, MTBF and usage rate are shown in Figure 6-1. The 
figure shows the positive error boundary for manning on the vertical axis as a 
function of various LCC input values on the horizontal axis. The closer the 
plot of each variable is to the horizontal axis, the less responsive manning 
is to variations in that variable. Reading the figure, the farther to the 
right a line intersects the +15% line, thd greater the error in that variable 
necessary to produce the fixed error in manning. 

The most powerful variable is usage rate because this simply acts as a 
scalar against all demand rates. Failure rate (the inverse of mean time 
between failure) is the next most powerful determinant, driving manning to its 
lower boundary at half its baseline value. The two maintainability variables 
are the least powerful, mean time to remove and replace having a greater 
effect than mean time to repair. This is explained by the fact that in the 
basic data set, the latter is only one hour while the former is three hours. 
This difference is accentuated by the fact that four of the LRU are coded 
discard at failure, which means that they will be removed and replaced but not 
repaired . 

Figure 6-2 is a plot of life-cycle cost variation against manning levels 
produced by changes in the four major variables. It is clear that usage rates 
have a strong effect throughout the cost estimate while reliability and 
maintainability statistics have a much smaller influence. Notice also that 
mean time to repair, even when driven to zero, cannot produce the -15% manning 
error . 


Another comparison of value is the different effect of MTBF errors on 
manning and spares. The error bounds on spares are reached at 86.7% and 
118.5% of baseline MTBF. The corresponding manning error, however, is only 
4.7% and -4.6% at these limits. To reach the J^15% manning errors, the 
multiples required are 66.8% and 200% of baseline MTBF. These relationships 
are illustrated in Figure 6-3. Thus, it appears that for this particular 
case, reasonable scrutiny of the spares cost estimate (and hence reliability) 
would be likely to confine the value of the manning error well within 
prescribed limits. Similar comparisons cannot be made for the maintainability 
variables because they do not affect readiness-critical outputs other than 
manning and training. 

The remaining possible source of excess error in manning estimates is 
from joint errors in input variables. This problem can be illustrated by 
joint errors in maintainability and reliability data. MTRR is used for the 
former and MTBF for the latter. 

Figure 6-4 illustrates the accelerated effect of joint errors in MTBF 
and MTRR in the presence of errors in the manning variable. The two graphs 
are the locus of manning errors produced by different levels of error in MTRR, 
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Figure 6-4. Relationship of Joint Errors to Manning 
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given a certain error has already occurred in MTBF. The graphs have been 
drawn for those values of MTBF that produced the upper and lower boundary 
errors in spares estimates. 

■ Notice that even under the joint error condition, a relatively large 
error in MTRR is required to produce a 15% error in manning. For an MTBF 
(118.5% of the baseline) that produces a 15% underestimate in spares, MTRR 
must also be understated by almost fifty percent (0.542) of its true value to 
produce a manning estimate 15% lower than its true value. Under the same MTBF 
error an overestimate of 87.6% in MTRR is required to produce a 15% overstate- 
ment of manning requirement. For an understated mean time between failure 
(86.7% of the baseline), the MTRR must be misestimated at about two thirds its 
value (0.37) and almost a third again its real value (133.9%) to produce mann- 
ing estimates off by minus and plus 15%, respectively. This one example sug- 
gests that manning may not be a particularly sensitive variable in comparison 
to other dependent LCC variables. 


F. RELIABILITY IMPLICATIONS 

1. Internal Reliability 

The four major findings of the SINCGARS replication were as 

follows : 

(1) There were problems in understanding how to construct the 
reference system. 

(2) There was confusion over the selection and use of design dif- 
ference indexes, perturbation values, and indirect productivity 
factors . 

(3) There was a lack of understanding of MOS/skill-level determination. 

(4) It was apparent that this type of analysis should be performed by 
a multidisciplinary team, not one analyst. 

Among the causes for these findings, beyond the general problems of lack 
of methodology clarity and inaccuracies caused by not having the technique 
automated, were problems with the instructions, insufficient or missing supple- 
mental input data, and excess emphasis on calculations instead of reasoned 
judgments. It was also apparent that the replication deviated too much from 
the training course content. Additional constraints included incorrect answer 
sheets upon which later analyses were based, syntax differences in the subjec- 
tive work packages, and the tedium brought on by the enormous quantity of 
answers required by Work Packages 4 and 5. Interestingly, the average number 
of answers that matched the HCM design team was high (78% in Package 4, 70% in 
Package 6) in some of the packages. However, in the all-important Data Pack- 
age 8, only 32% of the answers matched the corrected contractor response. 

This is a function of both the HCM clarity problem and the snowballing effect 
of previous administrative irregularities already present in the inputs of 
Package 8. Even with all the problems previously discussed, the study group 
appeared to understand how to apply HARDMAN to SINCGARS within the regulated 
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atmosphere of the replication. This finding was derived from the reasonably 
good performance on roughly half the data packages and the apparent improve- 
ment in methodology comprehension, once the discrepancies review conference 
was held. It is probably only now that the designers understand all of the 
problems that existed within the replication. If the original replication 
were used as a pilot study, a comprehensive and consistent reliability check 
could be performed. 


2. Qualitative Accuracy Study 

It seems that some of military contacts who deal with manpower and 
training have somewhat consistent opinions with the results of the face 
reliability and replication analyses. That is, the methodology is complicated 
and hard to understand, but when understood and applied correctly, has the 
potential to provide reasonable manpower and training forecasts for new 
procurements. This difficulty in understanding HARDMAN negatively influenced 
some of the personnel queried, or else others surmounted this obstacle and 
were able to make good use of the technique. It can be conjectured here that 
the comprehension difficulty experienced by the personnel contacted is similar 
to the problems encountered during the replication experiment. Once the 
methodological loose ends are corrected, it appears that HARDMAN could have 
widespread acceptance. 


3. Manpower Sensitivity Study 

The sensitivity analysis suggests that manning estimates may be 
fairly robust; they may not be as responsive to errors in input variables as 
the other important LCC elements. Because several of these quantities, such 
as MTTR and MTBF, are scrutinized for accuracy during the review process, and 
because they are more responsive, it appears likely that errors in inputs will 
be detected through them before they drive manning estimates out of the 
permissable boundaries. 

With regard to the data that drive these estimates, the results suggest 
that it would be unwise to devote extensive resources to their refinement for 
the purpose of improving manning estimates. The more serious sources of 
significant error in manning estimates that can be isolated from this study 
are the inappropriate selection of an operating scenario or deployment plan. 
The TRADOC and DARCOM communities control cost and manning estimates to a far 
greater extent than normally realized. By setting the number to be purchased, 
the number of operating units, and the hours of operation for the equipment, 
the program offices have the largest single effect on estimated quantities of 
anyone in the design process. Because early in the acquisition process these 
values are frequently not known with any precision, they are often considered 
unimportant when, in fact, they are pivotal LCC elements. For example, a 
system of which 100 will be purchased should be designed and built differently 
from one where 10,000 will be built. As a second example, usage rate dif- 
ferences of plus and + 13% will produce error in manning estimates +15%. 

These can be produced through an error in the estimated hours of opera- 
tion per year or in the number of units to be purchased. If both are off 
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by 6.3% or more, then the joint effect on usage rate will be 15% or more and 
the manning error will exceed the permissable limits. These remarks should be 
taken as a caution to program offices to devote the time and energy necessary 
to establish reasonable, stable values for these variables. Understanding 
that the sensitivity analysis provided here is somewhat limited by (1) the 
sample size and variety of acquisitions studied and (2) assumptions concerning 
the use of Navy billet data as representive of the Army billet structure, the 
findings are significant enough to warrant repeating the analysis once the 
Army has its own billet costing system completed. 
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SECTION VII 


CONCLUSIONS AND RECOMMENDATIONS 


A. OVERVIEW 

The preceding sections provided the findings in the four major validity 
areas of face, internal reliability, event-series, and qualitative accuracy. 

At the end of each section, the validity implications were assessed, along 
with supporting rationale. In this last section, the findings are first 
pooled and then summarized to illustrate the strengths and common major 
weaknesses that became apparent after examining and comparing all the validity 
implications. Next, the resolution of these common problem areas is addressed 
within the context of (1) the ranking of the weaknesses as a function of the 
user requirements and their impacts on the accuracy of the methodology; (2) 
the practical aspects of improving weak areas in terms of difficulty and 
potential for other organizations besides the original designer to adopt the 
technique; and (3) the reasonably low requirement to have extremely accurate 
MPT projections. Ultimately, a condensed, prioritized list of weaknesses and 
recommended corrective actions is provided. Additionally, follow-on study 
areas deemed important to closing out issues that could not be addressed 
within the time and budget constraints of this evaluation are also recommended. 


B. SUMMARY OF RESULTS 

In total, the HCM proved to be reasonably sound. Table 7-1 summarizes 
the strengths. The net strengths were based primarily on the high degree of 
conformance with the user requirements, the reasonable similarity to other MPT 
modeling processes, and the closure of eighty-five percent of the audit 
issues. It was also promising to discover that the projections did not have 
to be extremely accurate, considering the remaining twelve to fourteen problem 
areas that were found to potentially affect the accuracy and credibility of 
the projections. The potential concerns or problem areas uncovered, which 
generally spanned all the user requirements, operational, and reliability 
analyses, are shown in Table 7-2. 


C. PRIORITIZATION OF VALIDITY ISSUES 

Before any recommended revisions to the methodology can be made, it is 
essential to examine each of the issues from Table 7-2 and weigh their 
importance to meeting the Army's MPT needs and their implications relative to 
effecting the accuracy and usefulness of the projections. It is important to 
recognize in the near term that the methodology is reasonably sound. It is 
equally important to know the major limitations and the supporting fact that 
allowances can temporarily be made for some of the limitations in light of the 
inherent acceptable error bounds on the manpower projection. For the near and 
intermediate terms it would be useful to improve those areas most important 
and amenable to immediate correction, followed by a plan for improving the 
remaining far-term areas, perhaps requiring more supporting data. 
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Table 7-1. Net Results of HARDMAN Evaluation 


Evaluation Area 

Net Results 

User Requirements 

Technique complied with majority of 
mandatory user needs 

Operational Analysis 
Comparison with other 
MPT methods 

Methodology conformed with other known, 
accepted MPT modeling schemes and data 
foundations 

Audits 

Methodology demonstrated sound logic and 
reasonable results for 85% of test issues 
examined (remaining issues considered 
reparable in near term) 

Reliability Analysis 
Internal reliability 

Test group correctly replicated half of 
selected test points (two of four remaining 
test points found to be repeatable after 
clarification) 

Qualitative 
Accuracy check 

Two of nine individuals having experience 
with HARDMAN applications indicated a rough 
accuracy of 80-90% with actual manning 
requirements; remaining individuals had 
insufficient experience to comment 

Event Series Validity 

Manpower variable found to be least 
sensitive of all life cycle cost variables, 
demonstrating good utility of existing 
methodology even without near-term 
improvements 


7-2 





Table 7-2. Common Validity Issues 


Methodology Step 


Validity Issues 


Step 1 Data base construction 
and functional requirements 


Step 2 Build reference system 
and establish manpower/ 
skill requirements 


Step 3 Determine training 
requirements 


No firm criteria for selecting, altering, 
manipulating, or rating data sources 

Unclear indication of functional- 
requirement level of indenture 

No firm guidelines for constructing 
reference system (particularly when modeling 
two widely varying proposed designs) 

No consideration given to observed reduced 
manpower at end points of life cycle 

No firm range and criteria for selecting 
design difference indices (perturbation 
values), induced failure factors, or 
indirect productivity factors 

No complete means provided for equally 
comparing reference and proposed 
technologies 

Incomplete guidelines provided for 
selecting MOS/skills 

No quantitative means of differentiating 
between critical/non-critical tasks and 
resultant impact on training 


Steps 4 and 5 Determine 
personnel requirements/ 
shortages 


Projection does not consider manpower 
available at actual time of system 
deployment 


Step 6 Perform tradeoff 

analyses and present results 


Overall methodology 


Unstructured tradeoff process 

No indication provided for either relative 
credibility of projections or best 
estimate for manpower planning 

Incomplete life-cycle cost analysis 

No standardized assessment of study 
weaknesses and proposed resolution actions 

Low usability by anyone other than a 
well-seasoned, multidisciplinary team 
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Near-term Issues 


The issues having both sizeable user value and accuracy 
implications, and jointly found amenable to solution within the immediate 
one-year time frame were as follows: 

(1) No standardized criteria for selecting, altering, manipulating, or 
rating data sources. 

(2) Lack of proper level of indenture for functional requirements. 

(3) Insufficient guidelines for constructing reference system. 

(4) No consideration for reduced manpower demands at system life-cycle 
end points. 

(5) Unequal comparison of perturbed state-of-the-art technology 
against new proposed designs. 

(6) Lack of guidelines for selecting MOS and skills. 

(7) Unstructured tradeoff process. 

(8) No credibility indication of projections. 

(9) Non-standardized assessment of study weaknesses and resolution 
actions . 

Logic and credibility issues such as items (1), (3), and (5) above, are 
probably most important because they affect both the accuracy and usefulness 
of the methodology. Although removal of these problem areas would not 
guarantee improvement in absolute accuracy, it would certainly provide the 
users with a better subjective, relative indication of accuracy to aid 
decisions on budgeting of additional hardware tests or favoring one projection 
over another for planning purposes until additional information is received. 

All of these items were considered candidates for immediate resolution 
because elements of the solution for each of the issues often already existed 
in both the HCM design team and the Navy concerning the relative quality of 
various data sources (31, 40). For example, actual field or controlled 
collection data probably ranks highest on the list of preferred information 
for modeling new system performance because of its maturity, followed by 
operational or field-test data (reasonably mature), development test data, and 
free-flow or semi-controlled data collection information (limited sample 
size); and finally, contractor estimates (no test or sample data). 

As with the inherent experience available on data sources, it was 
discovered that solution structures similarly existed for items (3) and (5) 
above. The use of two reference systems to model two widely varying 
contractor designs was introduced earlier in Section V-F (the SINCGARS 
audit). Because the structure already existed for perturbing historical 
predecessor component data to fill data gaps in the proposed system, it 
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appeared that a simple extension of the same procedure would allow a 
"top-level" comparison of the total predecessor and proposed systems on an 
equal basis. 

The next near-term priority closely associated with credibility is the 
resolution of potential error sources. Items (2), (4), and (6) serve as 
potential error sources and therefore have accuracy and credibility 
implications. As indicated in the preceding paragraphs, it was learned that 
much of the solution framework already existed within the joint experience of 
the HCM design team who normally perform the HARDMAN analysis (34). The 
reduced end-point manpower demand issue, item (4), was considered a fairly 
simple error source to correct, considering that the deployment period and 
life cycle could be broken down into yearly time periods to be sensitive to 
the gradual buildup or tapering off of systems. Item (6) above, was another 
near-term issue that was found easily rectified based on knowledge obtained at 
both the audits and discrepancy review. 

The final near-term priority area is the resolution of some of the 
ambiguity issues. Items (7) and (9) are somewhat linked together and are 
rated as the last priority because their solutions serve more to improve the 
clarity of the process and projections than to contribute directly to accuracy 
and credibiity. Again, because the solution components for listing all 
potential tradeoffs, weaknesses, and proposed actions already existed to 
varying degrees, it was felt that these adjustments to the methodology could 
be a simple enhancement to the decision-making quality of the technique. 


2. Intermediate-term Issues 

Some of the validity issues discovered during the evaluation were 
found to have accuracy implications, but, upon closer examination during the 
audits and SINCGARS discrepancy review, were also found to be not amenable to 
immediate solution. Although some data and models were available and appeared 
to represent means of closing the issues, it was obvious that a certain amount 
of additional research and methodology restructuring were required and would 
probably consume one to two years of effort. The issues from Table 7-1 that 
are relevant to this intermediate category are as follows: 

(1) No firm range and criteria for selecting perturbation values, 
induced failure factors, or indirect productivity factors. 

(2) No consideration for manpower availability at the projected system 
deployment date. 

(3) Incomplete life-cyle cost analysis. 

In line with the reasoning provided earlier, items (2) and (3) above 
would have to be given highest priority because they both impact the accuracy 
and credibility of the projections. For both of these issues modeling 
processes, introduced earlier in Section V, have already been developed and 
with slight redesign could be incorporated in the existing methodology. Item 
(1) is fairly important because it could represent a major error source. Of 
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the three different types of factors, the perturbation value is probably the 
most crucial element. Some information presented during the SINCGARS 
discrepancy review is available to help define the problem of determining what 
can reasonably be expected regarding improvements in new technology 
performance. The induced failure and indirect productivity factors were 
considered slightly larger problems because it is difficult to determine when 
mistakes in selection of an arbitrary value for either one will generate a 
serious error in the outcome. 


3. Far-term Issues 

Some of the validity issues uncovered were found to require a 
substantial amount of data and time (probably about three to five years) to 
resolve them. In most cases the issues, although important, only surfaced in 
either the user requirements or the operational analysis and were therefore of 
a singular nature. The issues fitting in this category were as follows: 

(1) No quantitative differentiation between critical and non-critical 
tasks. 

(2) Incomplete consideration of demographic data such as age, sex, or 
education. 

(3) No consideration of the manpower impacts of other emerging systems. 

(4) No consideration of major socioeconomic fluctuations on manpower 
availability. 

Probably the largest potential error source is item (3) because the 
overall acquisition and planning process for new systems spans several years. 
Consequently, new concepts being approved have a very good chance of 
overlapping each other in their respective design and support planning 
stages. The remaining items are fairly equivalent in their level of 
importance. 

Great care was taken in the preceding paragraphs to refrain from 
including suggestions for improvements to concentrate on presenting a clear 
rationale for the near-, intermediate-, and far-term ordering of the various 
problem areas. Having prioritized all the various validity issues as a 
function of their potential for weakening the methodology and ease of 
solution, the final step is to explore sensible solutions. 


D. RECOMMENDATIONS 

One way to develop a plan for implementing recommendations is to first 
convert each issue into a positive proposed action and then summarize the 
total scope of intended actions. Table 7-3 first provides a summary plan for 
improving the existing methodology, followed by a detailed discussion of each 
suggested corrective action. 
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Table 7-3. Recommended Plan for Improving HARDMAN Methodology 



Potential 

for Improving Methodology 

Prioritized List 

Near-Term 

Intermediate 

Far-Term 

of Recommendations 

High Med. 

High Med. 

High Med. 


Establish criteria for rating, X 

selecting, altering, or 
manipulating data 

Provide indication of credibility X 

of projections 

Establish top-level method for X 

comparing technology of total 
reference/baseline and proposed 
systems 


Provide better guidelines for X 

constructing reference systems 

Firm up guidelines for functional- X 

requirements level of indenture 

Provide a better structure for MOS X 
and skill selection 

Reduce MACRIT manpower projection X 


down to one- or two-year projec- 
tions to be sensitive to lower 
end-point demands 

Provide a more structured X 

tradeoff process 

Establish standardized assess- X 

ment of study weaknesses and 
resolution actions 

Project manpower availability X 

out to actual system deployment 

date 


Provide more complete lift-cycle 
cost/ tradeoff analysis 


X 




Table 7-3. (Cont'd) 



Potential 

for Improving Methodology 

Prioritized List 

Near-Term 

Intermediate 

Far-Term 

of Recommendations 

High Med. 

High Med. 

High Med. 

Firm up ranges for perturbation 
values, induced failure, and 
indirect productivity factors 


X 


Incorporate potential manpower 
impacts of other emerging 
systems 



X 

Provide quantitative 
differentiation betwen critical 
and non-critical tasks 



X 

Incorporate more demographic 
data into personnel/availability 
projections 



X 

Consider major socioeconomic 
fluctuations in manpower 
availability projection 



X 


The following paragraphs develop each of the recommendations in 
Table 7-3 by suggesting possible corrective actions. First, consideration is 
given to the methodology design, followed respectively by suggested training 
for potential HARDMAN analysts as well as follow-on research. 


1. Methodology Design 

Because the key credibility issues were primarily centered in the 
near- and interraedite-term time frames, the following discussion focuses on 
those recommendations that fit in that one- to two-year time span. Far-term 
corrective actions are discussed in the "f ollow-on" portion of this subsection. 


a. Suggested Corrective Actions for Data Management and 
Credibility Indication . It is generally the intent of any modeling technique 
to act as a guide or tool to assist decision makers. As suggested earlier in 
Section IV, the Army users indicated a need to be able to apply HARDMAN as an 
assist to the ASARC or program management budget decisions. Paramount to 
making program direction, budget priority, and type manpower decisions is 
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understanding both the kinds and quality of data employed in the analysis. 
Consequently, a simple data rating matrix is suggested as a means of both 
summarizing the net credibility of the projections in the executive summary of 
HARDMAN applications as well as aiding the construction of the consolidated 
data base in Step 1 of the methodology. Table 7-4 is provided as an example. 

Table 7-4 basically applies to the proposed design. The listings of 
both the weighting criteria and data source quality were based on the joint 
experience of the Navy and contractor HARDMAN analysts. The table has 
considerable flexibility in that the analyst can fill it out solely for the 
proposed system if the predecessor reference or baseline system data are good 
or evaluate both the reference and proposed systems separately if each con- 
tains suspect information. Similarly, the data evaluation can be done on a 
component-by-component basis. In all these schemes the analyst merely has to 
indicate the availability, maturity, and type of data employed that auto- 
matically places the outcomes in a low, marginal, or good credibility 
quadrant. Once the data source(s) has been located, the remaining table 
elements can be answered. Probably the final credibility assessment will span 
two quadrants (i.e., low to marginal or perhaps marginal to good). Ultimately, 
a summary table such as Table 7-4 could provide an overall assessment of the 
data and projections to the decision makers so that MPT planning and budgeting 
can always be supported by the more reliable projection throughout the system 
development phases. 

Processes for changing or manipulating data were also recommended as 
part of the suggested improvements. Although not as complicated as the 
data-rating scheme, a simple delineation of helpful hints for when to alter 
data inputs (along with supporting examples) is necessary to help clarify the 
data management process in Step 1 and improve the audit trail. Based on the 
findings of the previously described audits, the basic guidelines for handling 
data appear to be as follows: 

(1) When building the consolidated data base, first check the proposed 
system operator and maintenance data for completeness; if data are 
missing for any components, select appropriate component data from 
a similar existing system, perturb the data per instructions 
provided in Step 2 of the methodology, and complete the proposed 
system data set (see the SINCGARS HARDMAN results for an example) 
(28). 

(2) All proposed system data inputs should be filtered by a group of 
experts to establish confidence that the data are reasonable. For 
example, basic maintenance repair times should be checked to 
determine if sufficient time is allowed for obtaining test 
hardware and tools by comparing them against similar tasks on 
predecessor components. If erroneous failure rate, indirect 
productivity, or induced failure data are encountered, replace the 
data with comparable historical values (best choice) or select 
appropriate values from the original minimum performance criteria 
stipulated in the request for proposal, RFP (next best choice) 

(see the RPV HARDMAN results for an example) (29). 
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Evolutionary change in tech- 
nology and maintenance philosophy 






(3) Where necessary, it sometimes saves time to use more recent 
historical data in the data base because it is easier to collect. 
For example, if the more recent data are component test data, then 
a sample sensitivity study should be conducted to determine 
whether the less mature (but easier to obtain) test data produces 
significantly different projections than the older, more mature 
(but harder to gather) historical data. If not, then replace the 
historical component data with the comparable test data where 
appropriate (see the RPV HARDMAN results for an example) (29). 

(4) Occasionally, it may be difficult to find a predecessor duplicate 
of a new proposed component. In this situation a larger or 
smaller scale component with similar design characteristics may be 
used in the consolidated data base as a last resort. However, it 
is extremely important to rate the credibility of these data in 
terms of both operational similarities and environment (see RPV 
HARDMAN results for an example) (29). 

» 

b . Suggested Corrective Actions for Perturbing Reference System 
to Model New Proposed Design . A framework for perturbing predecessor data to 
fill gaps in the proposed system data base was summarized earlier in the 
audits. To provide the decision makers with a better means of evaluating the 
credibility of contractors' projections, it is necessary to compare both the 
total reference and proposed designs on an equivalent technology basis. In 
support of the "evolutionary" technology growth concept, several studies have 
been done which suggest that industry is often too optimistic about new 
technology performance (42). Regardless of the fact that much of the overly 
inflated performance is sometimes derived from inadequate testing, it seems 
that a 20% improvement in reliability or performance can reasonably be 
expected (41,42). With respect to Step 2 of the methodology, a simple 
top-level adjustment of the predecessor reliability and maintenance data might 
be structured according to the following guidelines: 

(1) If the design difference indicates a positive improvement and the 
operating environment for the new component is not as harsh as the 
predecessor component, then employ a 20% increase in reliability 
with a "less than" or "maximum" label on the manpower projection 
(e.g., < 5,000 operators required). 

(2) If the design difference indicates a positive improvement and the 
operating environments are essentially the same, increase the 
component reliability by 20%. 

(3) If the design difference is insignificant, or the positive design 
improvement is offset by a harsher operating environment, then 
leave the reliability and maintenance data the same. 
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(4) If the operating environment is considerably harsher than the 

predecessor system and the design and functional differences are 
insignificant, then do not perturb the predecessor data, and 
indicate with good confidence that the resulting manpower 
projections represent a minimum (e.g., a minimum of 5000 operators 
are required; or, the operator requirement is > 5000 people). 

With these indications on the manpower projection for the updated 
predecessor or baseline design, one can start to identify whether the 
contractor projection for new technology performance is reasonable because the 
projections will either converge or diverge. Along with the earlier credi- 
bility ratings on the various kinds of data, the decision makers have a better 
means of making budget decisions in support of the revised predecessor data or 
the contractor projection. 


c. Suggested Corrective Actions for Constructing the Reference 
System . During the audits it was discovered that, the HCM designers used a 
more sophisticated way of selecting predecessor components to build the 
reference system than explained in the methodology. The following additional 
steps are suggested to help clarify and better structure the Step 1 reference 
system construction process: 

(1) Include a complete listing of documents that help select 
predecessor components similar to the proposed system (e.g., U.S. 
Army Field Manual 24-24, Washington, D.C., 20 May 1977; Department 
of Army (DOA), Aviation Electronics Configuration Directory, 
December 1970). 

(2) Include a listing of typical Army and industry technical contacts 
(by organization and division) in various component classifications 
(e.g., radios, surveillance sensing, etc.) that can confirm 
similarities or differences between the reference and proposed 
systems. 

(3) Eknploy two reference systems if two proposed systems, although 
functionally similar, employ completely different technologies 
(e.g., a Howitzer versus a rocket launcher). 

d. Suggested Corrective Actions for Functional-Requirements Level of 
Indenture . Similar to the preceding reference system suggestions, the audits 
revealed that the HCM designers employed a decision process which, although 
simple, provided more functional and task requirements guidance than the 
published methodology. It is suggested that the following steps be included 
in Step 1 of the methodology to better define the level of indenture of the 
functional requirements: 

(1) Go to a level in which the associated component tasks stop 

branching off (i.e., new tasks requiring different people cannot 
be identified, such as a mechanic who can repair both the truck 
engine and the chassis). 
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(2) Go to a level in which no further technology differences can be 
delineated between components. 

(3) Go to the same level as available sample predecessor component 
maintenance data in the consolidated data base. 


e. Suggested Corrective Action for MOS and Skill Selections . 

The audits discovered that the cookbook approach outlined in the methodology 
was not complete. The additional step not included in the methodology, which 
ensures the proper matchup between tasks and skills, is the use of experts 
from various proponent schools. Therefore, it is suggested that the 
methodology include a list of Army proponent school contacts (for all types of 
component groups and technologies) that can confirm or alter the task and 
skill selections as necessary. 


f . Suggested Corrective Action for Potential End-Point Manpower 
Errors . The solution to the end-point manpower shortage problem was outlined 
earlier in this section. The basic solution revolved around shortening the 
intervals between manpower projections, particularly when considering manpower 
demands of emerging systems during the early acquisition or later life cycle 
phases. Figure 7-1 graphically displays the solution by showing MACRIT pro- 
jections at one-year increments during periods of rapid changes in deployment 
and attrition, and the typical long term MACRIT rectangular projection for the 
mid-cycle, steady-state period. 


g. Suggested Corrective Action for Structured Tradeoff 
Process . To instill confidence in the reader that the methodology encompasses 
all possible tradeoffs, a simple table could be constructed as shown in 
Table 7-5. Table 7-5 could be placed in the executive summary as a means of 
displaying (1) the range of potential tradeoffs that the methodology could 
provide and (2) the actual tradeoffs conducted for a particular application 


POTENTIAL ERROR 
INDUCED BY NOT 
SHORTENING PRO- 
JECTION INTERVAL 


EMERGING SYSTEMS 


EARLY 
ACQUISITION 
BUILDUP 



LIFE CYCLE, yr 


Figure 7-1. Revised MACRIT Manpower Projection 
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Table 7-5. HARDMAN Methodology Tradeoffs 



as constrained by time or the sponsoring agency. The actual tradeoffs 
explored would be indicated by placing X's in the appropriate slots on the 
matrix. It should be noted that Table 7-5 is also useful if it is desired to 
display a clean summary of the various parameter sensitivities. Typically, 
the sensitivities would be indicated by a relative percentage change (e.g., a 
1% decrease in reliability results in a 10% increase in manpower). These more 
detailed cost sensitivity relationships could be summarized by placing the 
percentage "deltas" in the appropriate matrix slot. For example, continuing 
with the reliability illustration, one could read horizontally across the 
matrix and immediately see the net impact on the other LCC variables. This 
more detailed cost-sensitivity matrix could be used to cite pivotal cost and 
tradeoff benefits and jointly close out Step 6, tradeoff considerations, of 
the methodology. 


h. Suggested Corrective Action for Summarizing Study 
Weaknesses . Although listing study weaknesses is a simple procedure, it is 
extremely important information for decision makers. In conjunction with a 


7-14 


systematic listing of weaknesses, it is also important to suggest possible 
resolutions. For example, it is perfectly feasible that both the reference 
and contractor manpower projections could have "low-to-marginal ,, credibility 
ratings assigned because of poor or insufficient data. Consequently, the data 
problems would be listed as weaknesses and a suggested resolution might be to 
fund test programs for the major components that are driving the projection 
credibility down. A simple, dual-heading table consisting of "Major Study 
Problem Areas" and "Suggested Resolution Actions" could be placed in the 
executive summary to assist the decision makers. This adjustment would be 
fairly minor because the HCM designers already address problem areas in some 
of their applications (see the HARDMAN SINCGARS application) (28). 


i . Suggested Corrective Action for Projecting Manpower Out to 
System Deployment . Examples of manpower regression models were provided 
earlier in Section V. The key problem identified with these models was the 
lack of any indication of uncertainty in the manpower availability 
projections. Nevertheless, it is extremely important to compare the projected 
manpower requirement against manpower available starting at the point of 
system deployment. A simple initial approach to this dilemma might be to 
employ a linear regression technique, such as Barnes' Soldier 90, and allow an 
end point spread representative of only 75 or 80% confidence (i.e., select a 
confidence interval low enough to prevent too large a spread at the projected 
system deployment time) (43). The final projection may appear as shown in 
Figure 7-2, where the shaded area might represent the zone of confidence in 
which 75 or 80% of the observed data points center around the least squares 
projection (43). 


If the projected manpower requirement resulted in the situation depicted 
in Figure 7-2, then obviously there is potential for a critical manpower 
shortage. However, if the projected requirement fell within the shaded area, 
then for purposes of early planning, it would appear that there 



Figure 7-2. Use of Confidence Intervals for Manpower Projections 
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was no immediate problem. The advantage of this approach is that the 
projection can be made with greater confidence as the deployment date is 
approached and more data points on manpower availability are obtained. 
Probably the actual confidence in the projection would not have to be any 
better than one minus the allowable error on the manpower projection (i.e., 
roughly 0.8, as determined in Section VI). 


j . Suggested Corrective Action for Improving LCC Tradeoff 
Analysis . As shown earlier in Section V, some life-cycle cost elements are 
already included in the HCM. One of many typical life-cycle cost models was 
also briefly summarized (i.e., HLCCM). It is merely suggested here that other 
life-cycle cost models be reviewed and the remaining missing cost parameters 
be incorporated in the methodology to form a complete cost and impacts 
assessment . 


k. Suggested Corrective Actions to Firm Up Perturbation, 

Induced Failure, and Indirect Productivity Value Ranges . The 20% upper bound 
on the expected performance and reliability improvement of new technology was 
discussed earlier in this section. This expected improvement, however, is 
only one part of the complete picture. The previous discussion also pointed 
out the possibility that a different operating scenario or harsher environment 
could drive the reliability of a new design below that of its predecessor. At 
a top level it was merely suggested that the final manpower projection be 
stated as "greater than X number of people." The next step would be to try to 
establish an approximate estimate of the potential drop in reliability. A 
threefold approach is suggested to obtain this estimate: 

(1) Studies such as the earlier referenced Dobbins paper suggest 
roughly an order of magnitude difference between contractor 
estimates of reliability and actual field operation (42). It may 
be enlightening first to determine the components involved in the 
above Dobbins reliability tests, choose equivalent predecessor 
components, compare the operating environments in terms of 
relative harshness, and calculate the incremental difference 
between the respective reliabilities to get an idea of whether new 
technology (operating in a different environment and adjusted to 
its realistic performance level according to Dobbins) experiences 
a reduction in reliability below its predecessor. 

(2) As a next step it may be useful to conduct a survey of program 
managers in all four services to gather examples (and maintenance 
data) of systems and components where the reliabilities fell far 
short of their original goals. Again, establish the equivalent 
predecessors that were replaced by those new designs, compare the 
harshness of the operating environments, and calculate the 
incremental difference in reliability between the two. Several 
different kinds of components should be examined to determine 
which technologies are more sensitive than others (i.e., 
computers, propulsion components, firing devices, etc.). 
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(3) Finally, establish a group of experts from various technical 

fields in DOD to adjust the estimates considering that research 
and development programs provide engineering fixes that allow some 
of the reliability to be regained before a fairly steady-state 
reliability level is reached. 

The induced failure and indirect productivity variables can effect the 
outcome of the manpower projection even though their ranges are not as broad 
as the perturbation value. As a first step it is suggested that a sensitivity 
analysis be conducted, using completed HARDMAN applications, by simply 
calculating the spread in manpower for the complete ranges of both variables. 
If the spread does not aggravate the sensitivity of the manpower variable, 
then a single, perhaps mid-point, value should be selected to remove any 
confusion in picking a value from the possible ranges. If the spread exceeds 
the allowable error, then more detailed breakdowns using industry and services 
industrial engineering studies (for indirect productivity factors) or the Navy 
200,000 Hour Test Study (for human induced failure rates) will have to be used 
(41). 


2. Training of HARDMAN Analysts 

One of the major results of the audits and reliability analyses 
was that the methodology appeared usable by only a select group of 
individuals. Comments made during the debriefing following the internal 
validity test universally suggested that no one individual could exercise the 
methodology reliably; a team effort was absolutely required. It was 
discovered that the net expertise required was a combination of the followings 

(1) Sufficient military experience to provide both an understanding of 
where to obtain training and technical information, as well as the 
structure of the military logistics support, personnel and 
training environments. 

(2) Technical capability in at least one of a wide range of technical 
fields encompassing mathematics (particularly modeling), 
behavioral sciences, aerospace engineering, mechanical 
engineering, electrical engineering, and chemical engineering. 

(3) Experience in computer information and data management systems. 

(4) Experience with DOD personnel assignment and tracking processes. 

Again, based on comments made by replication analysts during the 
debriefing and discrepancies review (41), it seems that the most suitable 
combination of the above abilities was either items (1), (2), and (3), or 
items (1), (2), and (4). Note that the joint military and technical 
experience recur in both combinations. The personnel experience could be an 
extension of the military experience; and, similarly, the computer information 
system experience could greatly enhance the technical experience. All of 
these attributes exist to varying degrees in the individuals employed by the 
HCM contractor who execute the HARDMAN methodology (31). While some of the 
first-year methodology design recommendations are being explored, the Army 
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might consider using these qualifications to select and staff its HARDMAN 
analyst team(s). In preparation for possibly conducting in-house applications 
of the methodology or managing and auditing contractor applications, it is 
extremely important that the Army firm up the various judgment areas per the 
preceding recommendations. This effort will also provide for a clearer 
training foundation. Drawing on both the results of the preceding evaluation 
and debriefing comments, the following training suggestions are provided: 

(1) The original objective of the HARDMAN training session (in 
preparation for the internal validity test) was to concentrate on 
developing skills for the judgmental portions of the methodology. 
This objective should remain the major training thrust because the 
algorithms and number manipulation are fairly straightforward. 
Comments made on the debriefing questionnaire suggest that the 
first training session concentrated too heavily on the numbers and 
data manipulation. 

(2) The handouts (particularly the LSA data and examples) should be 
designed and their presentation associated more clearly with each 
step and judgment in the process. The emphasis should be placed 
on defining the judgment process and, where calculations are 
required, concentrate on only a few select cases that demonstrate 
the possible range of judgments that may be encountered. Note 
that the suggested corrective actions made earlier in this section 
could help refine the judgment process and that some of the 
peculiar data management, reference system construction, and 
credibility problems discussed earlier as part of the SINCGARS, 
RPV, and DSWS audits could be employed as example problems. 

(3) Break down the process by areas of expertise and provide different 
focuses for the backgrounds of various individuals (i.e., teach 
the team approach and define participant responsibilities). 

(4) Train for the complete six-step methodology (e.g., as part of the 
training program it may be useful to have the Army analysts work 
as apprentices at the HCM contractor's facility as a means of 
obtaining experience). 

These recommendations should not be construed as only applying to 
HARDMAN analysts. Even if the Army only chooses to monitor contractors 
performing applications in response to a Data Item Description (DID) in a 
contract, it is still imperative that contract monitors have the recommended 
training to ensure quality control of the outcomes and answer ASARC or Program 
Office audit requests. 


3. Suggestions for Follow-on Research 

There were four far-term areas identified earlier in this 
section. These areas revolved around consideration of emerging systems 
(horizontal analysis), quantitative differentiation between critical and 
non-critical tasks, expansion of the demographic data base, and consideration 
of major socioeconomic fluctuations. Except for horizontal analysis, the 
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remaining areas are probably not critical to early Phase 0 and Phase I 
top-level budget planning. As a first step, it is suggested that the Army 
initiate appropriate directives to require early planning of "most likely" 
candidates to be approved as emerging systems, inclusive of early operational 
and projected logistics support data. It is recognized that this is a large 
and perhaps sketchy task. However, if the Army wishes to meet its user needs 
and consider major drains of other systems on the personnel pool, it must 
start the mechanism to gather the needed information as soon as possible. 

The three remaining areas are considered far-term because they represent 
large data-gathering activities. For example, the quantitative critical and 
non-critical task differentiation effort might be accomplished by (1) first 
establishing criteria for critical and non-critical tasks (e.g., life or 
mission endangering, task difficulty); (2) selecting an array of systems 
representative of obvious task extremes for operator or maintainer performance 
(e.g., aircraft pilot versus an RPV operator); (3) calculating the ratios of 
instructor hours for the extremes; and (4) developing a simple rule of thumb 
to increase instructor contact hours by "X" percent in Step 3 of the 
methodology for those tasks judged critical. The time-consuming aspect of an 
effort such as this is the task sorting and rating study associated with item 
(1) followed by the gathering of system examples and instructor task hours 
associated with items (2) and (3). It is for these reasons that the "obvious 
task extreme" qualifier was suggested as a starting point. 

The completion of the demographic data base is a massive effort because 
it not only involves the gathering of data on age, sex, education, etc., but 
also requires expansion of the decision network in Step 2 of the HCM (i.e., 
skill and M OS assignments ) . It may be worthwhile for the near and 
intermediate terms not only to require manufacturers contractually to adher 
strictly to accepted human engineering guidelines but also to establish a 
design auditing mechanism to ensure strict compliance with sound human 
engineering standards. 

The last area, socioeconomic impacts on manpower, should be pursued in 
the manner expressed by Enke earlier in the document. In the interim, a 
subjective weighting mechanism might be employed to indicate the general 
direction of manpower trends (up or down), based on expert experience with the 
various variables (such as the economy). Referring back to the earlier 
recommendation on the use of confidence intervals around the long-term 
manpower availability projection, the final manpower projection might simply 
indicate that the most likely availability population would be either above or 
below the least squares linear projection, depending on the historical effect 
of the socioeconomic variables. 

The final far-term research area refers to one of the original 
constraints imposed on this evaluation. Due to time and budget constraints, 
only a qualitative examination of methodology accuracy was feasible. It is 
strongly suggested that a feedback loop be constructed so that actual field 
data on completed applications are gathered as the new systems are deployed. 

If the actual system MPT performance falls outside the allowable error 
discussed in Section VI, then appropriate steps should be taken to backtrack 
through the studies and strengthen those weak areas in the methodology. 
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In conclusion, several different validity examinations were structured 
to evaluate the HCM. Table 7-1 indicates that the technique was generally 
sound and basically integrated all aspects of the HPT problem well. Although 
some weaknesses were discovered, solutions (or solution structures) were 
apparent in most cases and appeared attainable within a reasonable time 
period. If the Army chooses to pursue the proposed recommendations, several 
benefits might be realized: 

(1) The Army will move closer to developing its own in-house HARDMAN 
capability (if it elects to do so). 

(2) The Army will improve the audit trail on MPT projections. 

(3) The Army will have a sound tool for producing MPT requirements, 
weighing the credibility of projections, and making better 
personnel recruitment and budget decisions in the manpower support 
arena. 


7-20 



REFERENCES 


1. "Major System Acquisitions," Department of Defense Directive 5000.1, 

March 29, 1982. 

2. "Major System Acquisition Procedures," Department of Defense Directive 
5000.2, proposed revision, October 30, 1981. 

3. Ritchie, M., "Manpower Implications in the Design of Air Force Equip- 
ment," National Technical Information Service (NTIS), February 1974. 

4. Eckstrand, G., "Technology Projection: Manpower and Logistic Factors in 

Weapon System Development," NTIS, August 1980. 

5. Thomas, G., and Butler, R. "An Informal History and Critique of the 
Navy's HARDMAN Program," working paper, December 1982. 

6. HARDMAN Methodology Handbook , Dynamics Research Corporation, Vol. 1-3, 
November 1980. 

7. Training Requirements Determination Handbook , PACER Corp., Vol. 1-5, 

June 1982. 

8. Butler, R. , "HARDMAN Cost Model System," Assessment Group Co., Santa 
Monica, California, Volumes 1-3, August 1981. 

9. Rose, Col. Louis, "HARDMAN Data and ASARC I Boilerplate," Memorandum, 
Soldier Support Center, Soldier Development Directorate, March 23, 1983. 

10. Wildrick, Ma j . Edward III, "Industry Survey of MPT Needs," Briefing in 
Minutes from Tenth Meeting of Department of Defense, Human Factors 
Engineering Technical Advisory Group, El Paso, Texas, March 8-10, 1983. 

11. "Weapon System and Equipment Support Analysis," MIL-STD 1388A, November 
1981. 

12. "TOE Manpower Authorization Criteria (MACRIT)," AR570-2, December 6, 1974. 

13. Fong, F. K., et al., "Unmanned Spacecraft Cost Model," Air Force Space 
Division, June 1981. 

14. Gordon, P., et al., "A Review of Underground Mining Equipment Research 
and Development," JPL Document 5030-482, November 1, 1980. 

15. Quade, E., and Boucher, W., Systems Analysis and Policy Planning , 

Elsevier Publishing Co., New York, 1968. 

16. Putnam, G. H. , "Example of an Early Sizing, Cost, and Schedule Estimate 
for an Application Software System," Proceedings of Third Summer 
Software Engineering Workshop , NTIS, September 18, 1978. 


8-1 



17. Winston, M. , et al., "A Preliminary Report on the Early Comparability 
Analysis (ECA) Methodology," Soldier Support Center, Regional HQ, 
Alexandria, Virginia, undated. 

18. Enke, S., "Army Manpower Prediction System (AMPS), Final Phase 1 
Report," NTIS, February 15, 1973. 

19. Holz, B., "The ELIM-COMPLIP System of Manpower Planning Models, 

Vol. 3, User's Manual Part 2," NTIS, December 1973. 

20. Safeer, H., et al., "A Computer Model for Projecting Civilian and 
Military Manpower," NTIS, February 1966. 

21. Doong, L., "A Manpower Model to Determine Future Inputs to NEC335X and 
NEC336X Billets," NTIS, December 1981. 

22. Drummond, H. E., and White, F., "Sources of Entrant Requirements in 
Manpower Models," Growth and Change , Vol. 7, No. 3, NTIS, July 1976. 

23. Leeson, G., "A Projection Model for Hierarchical Manpower Systems," 
Journal of Operational Research Society , Vol. 31, No. 3, NTIS, 1980. 

24. Lindsay, W., et al, "Simple Regression Models for Estimating Future 
Enlistment and Reenlistment in Army Manpower Planning," NTIS, 

September 1970. 

25. Barnes, H. , et al., "Soldier 90 Demographic Data and Trends," NTIS, 
October 1982. 

26. Grissmer, D., et al., "DOD and Navy Manpower Supply Scenarios Through 
2000," MATHTECH Inc., NTIS, June 10, 1977. 

27. Liberati, G., "Technology for Acquiring Supportable Systems: 

Managerial Overview of Test and Evaluation," Westinghouse 
Integrated Logistics Suppport Div., May 1983. 

28. "Application of the HARDMAN Methodology to the Single Channel 
Ground-Airborne Radio System (SINCGARS)," Dynamics Research Corp., 

. Technical Report E-8735U, September 1983 (only results of Steps 1 and 2 
audited) . 

29. "Application of the. HARDMAN Methodology to the Army Remotely Piloted 
Vehicle (RPV)," Dynamics Research Corp., Technical Report R-408U. 

Vol. 1 and 2, January 11, 1983. 

30. "Application of the HARDMAN Methodology to the Division Support Weapon 
System (DSWS)," Dynamics Research Corp., Technical Report E-8247U, 

Vol. 2, December 1982. 

31. Zimmerman, W., and Franklin, M. , Technical Audit of RPV and DSWS HARDMAN 
Applications, Dynamics Research Corp., July 18-22, 1983. 

32. Joint JPL, Army Research Institute (ARl), and Soldier Support Center 
(SSC) SINCGARS Replication Effort (seven individuals), May 9-27, 1983. 


8-2 



33. Knight, A., Advanced Lightweight Torpedo (ALWT) Logistician, personal 
communication, November 17, 1983. 

34. Wright, J., Lt. Cdr., ALWT Integrated Logistics Manager, personal 
communication, October 11, 1983. 

35. Vroom, 0., ALWT/SIRCS MPT Logistics Element Manager, personal 
communication, October 6, 1983. 

36. Horn, G., Sgt., Submarine Combat System (SUBACTS) Training Manager, 
private communication, November 10, 1983. 

37. Fagae, R., Destroyer (DDGF) Personnel and Training Analyst, personal 
communication, October 11, 1983. 

38. Ritter, W., Capt. Flight Trainer (VTXTX) Project Manager, personal 
communication, November 17, 1983. 

39. Preidis, R. , Airforce Operations Research Analyst, ASSET, personal 
communication, November 30, 1983. 

40. Lugen, D., Lt. Cdr., personal communication, Assistant Manager, Navy 
HARDMAN Office, Washington, D.C., April 4-7, 1983. 

41. Joint meeting of JPL, Army Research Institute (ARl) , Soldier Support 
Center (SSC), and Dynamics Research Corporation (DRC), SINCGARS 
Discrepancies Review Conference, August 22-24, 1983. 

42. Dobbins, D. A., "The Dangers of Relying on Industry as a Partner in 
Material Development," Army Research, Development, and Acquisition 
Magazine , Vol. 23, No. 2, March-April 1982. 

43. Miller, I., and Freund, J., Probability and Statistics for Engineers , 
Prentice Hall Inc., New Jersey, (pp. 231-234) 1965. 


8-3 



APPENDIX A 


INTERVIEW QUESTIONS FOR USER REQUIREMENTS 
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USER REQUIREMENTS 
QUESTIONNAIRE 


1. To what organizational division of the Army do you belong? 
VERBATIM 


2. What is the primary charter of your organization? 
VERBATIM 


3. What are your specific duties within the organization? 
RESPONSIBILITY : (Areas of Concern, Deliverables, etc.) 


4. What are your key needs in making or using manpower personnel and training 
projections (i.e., what do you need to have to do this?) 

INPUTS ; Type of data. Level of Detail 


INTERFACE CONSIDERATIONS 

Other Organizational Actors (input/output ) 


Timing 


5. What are the key drivers in needing such MPT projections, e.g., cost, 
planning, safety, performance, readiness/force modernization, etc? 
SPECIFY AND EXPLAIN , if possible 


6. Which ARMY MPT regulations/documents pertain to your organization's 
involvement in the M.P.T. process? 


7* What technique is presently employed for getting MPT inputs into system 
acquisition process? 


8. Who has responsbility for this technique 1 1 

9. To your knowledge, does it provide good results? 

SPECIFY 


10. In your organization, who is (are) the most likely user(s) of a 
methodology to make MPT projections? 



11. What other users of the the methodology could you identify within the Army 
organizational structure? 


12. Who in your organization is (are) the most likely user(s) of the results 
of such a methodology? 


13. What other users of the results of such a methodology could you identify 
within the Army? 


14. What suggestions do you have for the design of a methodology to accomplish 
MPT projections? (OUTPUTS, SCHEDULE, PROCESS, ETC.) 


15. What would be the most useful format(s) for the results? 

16. At what point in the acquisition process ti.e. Phase 0, I, etc.) do you 
require MPT results for (a) design changes, or (b) cost control? 
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USERS 


REQUIREMENTS (DESIGN) 


1 MPT PROJECTIONS NEEDED BEFORE MILESTONE 1 OF SYSTEMS 
ACQUISITION CYCLE FOR MAXIMUM DESIGN INFLUENCE AND 
COST CONTROL 


2 MPT DATA SHOULD BE PROVIDED THROUGHOUT A SYSTEM'S 
LIFE CYCLE 


3 DATA SHOULD BE DEVELOPED FOR SYSTEM ALTERNATIVES 
(PROPOSED). AS WELL AS PREDECESSOR SYSTEM FOR COST 
COMPARISONS 


4- FEEDBACK MECHANISMS SHOULD BE INCORPORATED TO 
PERMIT DEVELOPMENT OF HISTORICAL DATA BASE WITH 
UPDATE CAPABILITY 


5 SYSTEM SHOULD BASE CALCULATIONS ON RELIABLE DATA 
SOURCES AND DOCUMENT THESE SOURCES FOR CREDIBILITY 


6 ISA DATA SHOULD BE UTILIZED AND PRODUCED IN LSA 
FORMATS FOR EASY RECOGNITIONIUSE 


7 MPT MODEL SHOULO EXPOSE ASSUMPTIONS ANO LIMITATIONS 
WHICH DISTINGUISH PROPOSED FROM CURRENT SYSTEM DATA 


8 MANAGEMENT INFORMATION SYSTEM SHOULO FOCUS ON 
"FIRST UNIT EQUIPPEO," INCLUDING SPACES. FACES. AND 
EOUIPMENT DATA 


9 MPT MODEL SHOULD BE TARGETED TO BOTH BUDGETARY AND 
SYSTEM MILESTONES ON A TIMELINE WHICH INCLUDES 
ACTION-SIGNALLING MECHANISMS AND SPECIFICATION OF 
RESPONSIBLE PARTIES 


10 PERMANENT. CENTRALIZED ARCHIVING CAPABILITY (HARDCOPY 
AND DISK STORAGE) IS ESSENTIAL FOR 
ACCESS 1 AUDIT) REVIEW! UPDATE 


11 FRONTEND COSTIBENEFIT CRITERIA SHOULD BE APPLIED 
BEFORE SELECTING SYSTEM FOR MPT ANALYSIS (BASEO ON 
TECHNOLOGY CHANGES) 


12 MPT MODEL SHOULO IDENTIFY SUPPORT SYSTEMIS) OR OTHER 
SYSTEMS WITH OVERLAPPING OBJECTIVES IN PROVIDING 
COMPREHENSIVE MPT ANALYSIS 


13- MPT DOCUMENTATION FORMATS (REPORTS SHOULD BE 
COMPATIBLE WITH THOSE USED TO INFLUENCE TRAOOC MOS 
DECISIONS 



14 MPT DOCUMENTATION SHOULD PROVIDE DCS PER WITH 
COMPARATIVE MANPOWER REQUIREMENTS FOR SYSTEM 
AUERNAT1VESIPREDECESSORS 


IS DOCUMENTATION MUST BE AVAILABLE FROM DCS PER AT FRONT 
END RE: AVAILABILITY OF PERSONNEL FOR EACH MOSISKIll 
LEVEL ASSOCIATED WITH SYSTEM 


IB TIME1YIACCURATE DATA MUST BE GIVEN TO CONTRACTORS TO 
PREPARE REP'S AND/OR ORGANIZATIONAL MAINTENANCE 
MANUALS 


17 IMPROVED VEHICLE FOR MPT INFORMATION EXCHANGE 
MANDATORY TO PERMIT EFFECTIVEI TIMEIY COMMUNICATION 
(SYSTEM SHOULD BE INTERACTIVE) 


18 MPT MODEL MUST MEET MULTIPLE USER'S NEEDS FOR 
OPERATIONAL FEASIBILITY & INTEGRATION INTO. ANDIOR 
IMPLEMENTATION IN THE ARMY THROUGH EXISTING REPORTING 
STRUCTURES 


HARDMAN USER REQUffiBIERTS SUMMARY MATRIX (DESIGN) 
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METHODOLOGY AUDIT 


Generic 


Steps 1 and 2 


1. What judgment determines the level of indenture necessary in determining 
the functional requirements? 

2. How does one determine the maturity of the data presented? 

How does on ensure selecting the most mature data source(s)? 

3. The judgments used to determine manpower drivers in the design 
difference index are not clear to me: 

How are the key drivers flagged? 

How is the perturbation value assigned? 

When and where is the perturbation value applied? 

4. What judgment is required and how is it applied to select the lowest 
skill level MOS for any maintenance task when the AR611-201 seems to 
define the performance of each MOS quite clearly? 

5. What judgments temper the use of the indirect productivity factor? Is 
this a blanket application or a selected application? Define why and 
when. 

6. What judgments allow the incorporation of two tasks, normally performed 
by two separate MOS's, into one MOS? 

7. What judgments are used in creating a new MOS? 

8. When and where are the perturbation values applied to the reference 
system to achieve a baseline system with which to compare the contractor- 
proposed data? 

9. Is the function to address the manpower loading a rectangular or weible 
curve? 

10. How is the accuracy of the reference system ensured when contractor- 
proposed data is used in building a reference system component for which 
no predecessor data is available? 

11. How do you construct a reference system to be similar to the proposed 
system when the proposed systems vary widely? 

12. Does the HARDMAN Methodology take into account factors other than those 
addressed by the contractor? Are the DRC reference system manpower 
requirements more credible than the contractor-proposed requirements? 
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METHODOLOGY AUDIT 


Generic 


Step 3 

1. Is the training necessary for the instructors taken into account? 
Training for the instructor’s instructors? 

2. Are the costs of training the instructors taken into account? The costs 
of training the instructor's instructors? 

3. What judgments are used in identifying and constructing training courses 
for any new ratings developed? 

4. When the technical factors for determining the baseline skill levels are 
analyzed, is memory decay taken into account? 

5. Explain the two levels of indenture for determining training 
requirements; the Methodology seems to address only the first. 

6. What determines the frequency or need for refresher (retraining) courses? 

7. How does the TRRA estimate the cost of devices required for training? 

8. Are the costs for personnel movement between training schools and 
training periods included in training costs? Are these large costs? 

9. How often is it required to send personnel back through the training 
schools? Is this a large cost? 

10. How are the perturbation values determined in steps 1 and 2 applied to 
the reference system training courses? 


C-4 



METHODOLOGY AUDIT 


Generic 


Step 4 

1. How are the personnel required for new MOS's accounted for? 

2. Where is the flow of second enlistment personnel taken into account? 

3. How is a personnel flow conducted for a new MOS when no historical data 
is available? 

4. Where do the probabilities to calculate the flow of personnel come 
from? How are they applied to the MOS populations to establish the 
expected values of availability? 

5. Are major socioeconomic fluctuations considered? 

6. Is a steady-state population within the Army assumed? 

7. How are the perturbation values determined in steps 1 and 2 applied to 
the reference system personnel requirements? 


Conclusions 

1. Why is a Life Cycle Costing (LCC) model not included in the Methodology? 

2. When, if at all, is a wartime scenario addressed? 

3. How much credibility is lent to the results of the analysis? 
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METHODOLOGY AUDIT 


DSWS/RPV-Specific 


Steps I and 2 
DSWS 

1. What justification does PACCAR have for utilizing an "M-2-like" chassis 
in a potential overload situation? 

2. What accounts for the vast differences in manpower between the Norden 
M109E4 system and the reference and other proposed systems? 

3. Why weren't personnel and training analyses performed on the PACCAR 
system? 

4. The Norden M109E4 system has no Ammunition Resupply Vehicle (ARV). The 
PACCAR system uses only one ARV per two Howitzers. How do these two 
systems get considerations as proposed systems if the functional 
requirements were developed properly? Do you consider the fact that the 
PACCAR system uses one ARV per two Howtizers a potential error source? 

5. Why was a shipboard gun autoloader chosen for the reference system? 

6. For the following subsystems, what judgments were used in determining 
the design differences? What judgments were used in assigning a 
perturbation value? How were these values applied to the reliability 
and maintainability data? How did they affect the manpower projections 
for the reference system? 

a. Autoloader 

b. Fire Control Processor 

c. Internal Navigation System 

d. Auxiliary Power Unit 

e. Suspension: 1. Howitzer 

2. ARV (where applicable) 

f. Howitzer Engine 

g. Suspension/ Chassis Combination 

7. In building the reference system, what criterion were used to determine 
the maturity of the data? 

8. How does the use of a trailer with the FMC system affect the reliability 
and maintainability of that system? How does the use of a trailer 
affect the munitions-loading capabilities of that system? 

9. What accounts for the vast differences in manpower projections between 
the reference and proposed systems in the following MOS categories: 

31V, 31E, 35E, 63D. 

10. What accounts for the extremely low number of the following MOS's in the 

Norden M109E4 system: 31V, 31E. 
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METHODOLOGY AUDIT 


DSWS/RPV-Specific Ccont'd.) 


Steps 1 and 2 

DSWS (cont'd.) 

11. Why was an airborne unit selected as the reference navigation system? 

12. What justification exists for Norden and FMC systems not using an 
Auxiliary Power Unit? 

13. How was the M-l tank Auxiliary Power Unit data altered to take into 
account its lack of predicted performance? 

RPV 

1. Track the source of the Mean-Time-To-Repair (MTTR) values of the 
reference system for the following subsystems: 

a. Data link terminal (airborne unit) 

b. Airframe assembly 

c . Launcher 

2. Identify the reference equipment and selection rationale for the 
following subsystems: 

a. Central Processor Unit (CPU) 

b. RPV control and display panel 

c. Airframe assembly 

d. AV engine 

e. Launcher 

3. Track the Contractor-Furnished Equipment data from the baseline system 
equipment that was used in the reference system for the following 
subsystems : 

a. Launcher 

b. CPU 

c. Airborne unit data terminal 

4. Identify the subsystems of the baseline system that did not meet the 
functional requirements. 

5. Identify the baseline equipment that used altered contractor data. 
Explain the rationale for altering this data. 
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METHODOLOGY AUDIT 


DSWS/RPV-Specif ic (cont'd.) 


Steps 1 and 2 

RPV (cont'd.) 

6. Identify and provide the rationale for the perturbation values (PV) 
assigned to the following subsystems: 

a. Airborne unit data terminal 

b. CPU 

c. AV airframe 

d. AV engine 

If no PV was used in the above subsystems, identify where the PV was 
used. 

7. Identify baseline equipment where no reliability and maintainability 
(R&M) data was available. 

8. Track MOS/Skill level assignments for the following subsystems: 

a. Airborne unit data terminal 

b. CPU 

c. Airframe assembly 

d. AV engine 

9. Page 48 of Volume 1, RPV states that the assumption is made that the MAC 
charts exclude the make-ready/put-away times, a value which would 
normally be adjusted by the Indirect Productivity Factor. In the 
SINCGARS replication, the instructions stated that the make-ready/ 
put-away times were included in the MAC chart data. Explain this 
discrepancy. 

10. Explain how full-sized aircraft data was altered in order to meet 
miniaturized aircraft data projections. 

11. Page 75 of Volume 1, RPV makes reference to certain modeling equations. 
Explain these. 

12. What is the basis for a two-to-one improvement of the M939 truck over 
the M809? 

13. What criterion are used in determining the number of systems to be 
fielded? Are these criterion adjusted for system availability? 
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METHODOLOGY AUDIT 


DSWS/RPV-Specific (cont'd.) 


Step 3 


DSWS 

1. How did the perturbation values assigned to components of the following 
subsystems of the reference system affect the training resources 
requirements analysis? 

a. Autoloader 

b. Fire Control Processor 

c. Internal Navigation System 

d. Auxiliary Power Unit 

e. Suspension: 1. Howitzer 

2. ARV (where applicable) 

f. Howitzer Engine 

g. Suspension/Chassis Combination 

2. What training impacts are effected by the use of a trailer in the FMC 
system? 

RPV : For Questions 1 and 2 

References: RPV Appendix C - pp. 3, 4, 27, 30, 37 

Subsystems: Air Vehicle, airborne data terminal, remote ground 

terminal 

1. What was the rationale for assigning the PV's? What are the values that 
were arrived at? 

2. What is the rationale for choosing the respective representative 
equipments for training estimation? What would the procedure be if a 
detailed Training Resources Requirements Analysis (TRRA) was conducted? 

3. Identify the nine new courses in the reference system; the seven 
modified courses in the baseline system; the two deleted courses in the 
baseline system. 

4. Discuss the implications of the fact that, in many cases. Direct Support 
(DS) level maintenance was not specified in the Logistics Support 
Analysis (LSA) data. 

5. Discuss the low performance that may result from high skill requirements 
and low paygrade assignments as indicated on page 119, RPV; explain why 
this was not identified in the recommendations ; how was this addressed 
in the reference system? 
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METHODOLOGY AUDIT 


DSWS/RPV-Specif ic (cont'd.) 


Step 3 (cont'd. ) 

RPV: (cont'd.) 

6. Explain, by example, the four steps in determining the number of 
instructors for one course. 

7. Show an example of a Program Of Instruction (POl) used for obtaining 
training man-days. 

8. Since the HARDMAN Methodology does not address Life Cycle Costing (LCC), 
and only addresses a potion of the training costs, why address it at 
all? Why wasn't it addressed in RPV? 

9. In general, how does this step interrelate with the MOS/ASI selection in 
step 1? 

Step 4 


DSWS 

1. How did the perturbation values assigned to components of the following 
subsystems of the reference system affect the personnel requirements? 

a. Autoloader 

b. Fire Control Processor 

c. Internal Navigation System 

d. Auxiliary Power Unit 

e. Suspension: 1. Howitzer 

2. ARV (where applicable) 

f. Howitzer Engine 

g. Suspension/Chassis Combination 

2. What personnel requirements are affected by the use of a trailer in the 
FMC system? 


RPV 

1. Does the primary output of this step, personnel to be trained per year, 
represent requirements at IOC, 5 yrs., 10 yrs., . . . ? 

2. Discuss the sources of personnel flow rates. Why does DRC rely on Army 
data and not develop its own models, or use the algorithms outlined in 
the HARDMAN Methodology handbook? What is the expected error in the 
primary output due to inadequate data inputs? 

3. Explain, by example, personnel flow in the 13T MOS structure. 

4. Are there other Army data sources available, such as TRADOC? 
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Step 5 


RPV 

1. Page 154, RPV states that meaningful results can be obtained in the case 
of RPV since it is a completely new system. Does this imply that 
meaningful results could not be obtained if this were not the case? 

2. Does DRC provide inputs or actually run the Army's Personnel Policy 
Project Model (P^M)? 

3. The P^M Model projects personnel suply based on Army internal 
variables; what of external variables such as the economy or politics? 

4. Are the P^M personnel inputs different from those used by DRC in step 
4 of the Methodology? 

5. Given the short horizon, one to three years, of the P^M Model, is 
there a better way and why? 

6. How do authorizations relate to the TO&E? 


Step 6 


RPV 

1. How long does it take to conduct a tradeoff study? 

2. Discuss, by example, the tradeoff study of the M939 truck versus the 
ML809 truck. 


Conclusion 


1 . 


What conclusion can you draw from the chart on page 5, RPV; what 
conclusions can we draw on how credible the answers are? 
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APPENDIX D 


DEBRIEFING QUESTIONNAIRE 
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TRAINEE DATA COLLECTION SHEET 


HARDMAN PROJECT 

In order to analyze and identify possible explanations for divergence in 
trainee's solutions to problems worked during the training course, we need to 
determine the following background data on each person. 

1. TRAINEE I.D. # (NAME) 

2. LEVEL OF EDUCATION 


NO. YEARS COMPLETED 

ADVANCED DEGREES? 

CERTIFICATES? 

COLLEGE DEGREE 

HIGH SCHOOL 

3. TYPE OF EDUCATION 

ADVANCED DEGREE FIELD(S) 

COLLEGE MAJOR 

H.S. EMPHASIS 

COLLEGE PREP _ 

TECHNICAL 

VOCATIONAL 

OTHER 

4. PRESENT EMPLOYMENT 


TITLE: 


EMPLOYER: 


DUTIES: 


LENGTH OF TIME (IF ANY) AWAY FROM PAST EDUCATION FIELD OF EMPHASIS 
(MAJOR, FIELD, ETC.) 

5. PAST EMPLOYMENT 


IN REVERSE CHRONOLOGICAL ORDER, PLEASE LIST RECENT JOB EXPERIENCE (UP 
TO 5 YEARS) 
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DEBRIEFING QUESTIONS 


I. Did you complete all 8 sections of the replication package? 
If NO, state which portion is incomplete... 


2. Were the recommended time frames realistic? 

If NO, state specific packages which tobk more/less time than recommended. 


3. Was the subject material or methodology of this replication process directly 
or indirectly related to your present job? 

If YES, state how (briefly) 
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4. Were there portions of the replication that you felt were particularly 

difficult? (Beginning with package 1, please give any significant notations 
you made about the package while working on the replication. Please omit 
problems which were readily resolved by a telephone call of DRC. Also in- 
clude comments about possible reasons and solutions for the problem.) 

Prompts: Complexity; Course training; Time factors, e.g., lag between course 

& replication or time to finish package; Materiels, Preparation of individual..) 
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5. How did you generally make corrections to your answer sheets on the KT Unit 
and the ECCM? 

Prompts: Straight DRC worksheet utilization; Mapping over of my answers 

to DRC worksheet or vice versa; Other ; Special problems noted. 
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6. Were the instructions in the data packages generally adequate for interpreting 
what was actually required, i.e., calculating the required answers? 

If NO, please explain possible reasons and state which data package. 


7. Did you note any significant descrepancles between your solutions and the 
"school" (DRC) solutions during the replication? 

If YES, please state reasons, if known. (Prompts: Replicator background; 

Training course; Data package instructions, materials; Other (errors, etc. ) 
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8. What would have been required of you to arlve at the school solution (DRC 
answers)? Apply comments to specific data packages. 


9. Did you read the pre-replication pafckage raateriels for the SINCCARS? 
If YES, were they useful? 


10. Did the course prepare you adequately for the replication? 
EXPLAIN. . . 


11. What criteria could you suggest for selecting analyst trainees to apply 
HARDMAN? (Answer may vary for different steps in the process) 


12. Do you have any suggestions for training course improvements? 
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DATA INPUTS FOR THE MANPOWER SENSITIVITY MODEL 
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This appendix contains the data sets and baseline runs of the SEDCOM 
cost model. The first table is the baseline run used in the analysis. The 
equipment used there is SINCGARS without the battery. This was done because 
the battery, with extremely low mean time between failures, takes a long time 
to run on the computer and adds nothing to the analysis. For the sake of 
completeness a second baseline cost estimate is presented, based on the 
inclusion of the battery. It is labeled SNCGARS2. 

Most of the input data used to describe both SINCGARS and its operating 
and support environments were provided by the SINCGARS Program Office and are 
presented in the next several tables. Because the computer program used for 
this example was derived from the Navy SECDOM model, the billet structure and 
some maintenance level terminology (for example, ship level is organizational 
level) were merely adopted to prevent reprogramming. The use of some Navy 
terminology and data did not affect the accuracy of the sensitivity study 
because this effort was conducted strictly as a top-level (order-of-magnitude) 
analysis. 
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SINCGARS 


********************************************************************************************* 

* COST ELEMENTS ($*000) * * SYSTEM RATES * 

******************************************* ********************************** 


Life Cycle Cost 

885323.0 

Number of LRA Types 17 

System Unit Cost 

11.1 

Number of LRA's 17 

INITIAL COSTS: 

Production 

359265.0 

Confidence Against Stockout .999 

Maint* Training 

1251.2 


Op. Training 

80850.0 

System MTBF 472.3 hrs. 

Misc« Manpower 

0.0 

System MTTR 3.0 hrs. 

Spares 

2844.2 


S+TE 

67048.6 


Tech. Doc. 

55.2 


**T0TAL 

511314.0 

SUPPORT POLICY SUMMARIES: 

LRA's coded COD 0 

OPERATION AND SUPPORT COSTS: 


LRA's coded MOD 9 

Maint. Wage 

31587.0 

LRA's coded local repair 4 

Op. + Off. Wage 

35684.7 

LRA's coded discard 4 

Maint. Training 

3842.2 


Op. Training 

248394.0 


Misc. Manpower 

0.0 

TRAINING BILLETS BY TYPE: 

Spares 

11018.2 

Equipment Operators 32340 

Repair 

2257.9 

Org and Int Maint 476 

Transportation 

20.2 

Depot Technicians 6 

S+TE 

41198.5 


Tech. Doc. 

5.5 

REAL MANNING REQUIREMENTS: 

**TOTAL 

374009.0 

Org. and Int. Maint. 235.97 


Depot Maintenance 4.92 

Operators 8085.00 


********************************************************************************************* 
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SNCCARS2 


********************************************************************************************* 


* COST ELEMENTS ($'000) 

* 

* SYSTEM RATES * 

******************************************* 

********************************** 

Life Cycle Cost 

.1126040.0 

Number of LRA Types 18 

System Unit Cost 

11.2 

Number of LRA's 18 

INITIAL COSTS: 

Production 

314545.0 

Confidence Against Stockout .000 

Malnt. Training 

1618.7 


Op. Training 

70280.0 

System MTBF 22.8 hrs. 

Misc. Manpower 

0.0 

System MTTR 0.4 hrs. 

Spares 

54779.7 


S+TE 

67048.6 


Tech. Doc. 

55.2 


**T0TAL 

508327.0 

SUPPORT POLICY SUMMARIES: 

LRA's coded COD 0 

OPERATION AND SUPPORT COSTS: 


LRA's coded MOD 9 

Malnt. Wage 

39707.1 

LRA's coded local repair 4 

Op. + Off. Wage 

31019.4 

LRA's coded discard 5 

Malnt. Training 

4971.3 


Op. Training 

215920.0 


Misc. Manpower 

0.0 

TRAINING BILLETS BY TYPE: 

Spares 

282912.0 

Equipment Operators 28112 

Repair 

1962.7 

Org and Int Malnt 616 

Transportation 

17.6 

Depot Technicians 6 

S+TE 

41198.5 


Tech. Doc. 

5.5 

REAL MANNING REQUIREMENTS: 

**total 

617714.0 

Org. and Int. Malnt* 296.63 


Depot Maintenance 4.27 

Operators 7028.00 


********************************************************************************************* 
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File name: SINCGARS 


********************* JVJAVY environmental and cost factors ********************* 

0 Daily cost of C-school training ($/day/student) 250 

1 Cost of A-school , maintenance technician ($/student) 0 

2 Cost of A-school, operator ($/student) 0 

3 Annual billet cost, maintenance technician 30583 

4 Annual billet cost, operator 28235 

5 Annual billet cost, maintenance technician at depot 38018^ 

6 Annual billet cost, officer 0 

7 Available weekly work hours, maintenance technician 48 

8 Available weekly work hours, operator 60 

9 Available weekly work hours, depot technician 40 

10 Annual turnover rate, shipboard personnel (decimal) 5 

11 Annual turnover rate, depot personnel (decimal) 33 

12 Annual discount rate (decimal) 1 

13 Condemnation rate (decimal) 01 

14 S+TE support cost : S+TE purchase cost (ratio) 1 

15 Cost of technical documentation development ($/page) 200 

16 Annual tech, documentation maintenance rate ($/page/yr) 20 

17 Cost of insured freight ( $/l b/mi 1 e ) 0002 

************************* system OPERATING ENVIRONMENT ************************* 

0 Number of ships on which system is deployed 28 

1 Number of systems per ship 1004 

2 Length of system life cycle (yrs) 10 

3 Length of deployment period (days) 5 

4 Number of deployments per year 52 

5 Average system operating hours (hr/wk) 15 

6 Peak system operating hours (hr/wk) 30 

7 Reduction rate (learning curve slope) 1 

8 Number of stockage depots 3 

9 Number of repair depots 2 

10 Cost of LRA repair at contractor depot 1000 

11 Ship response time (days) 2 

12 Military depot response time (days) 16 

13 Contractor operated depot response time (days) 20 

14 Distance between repair and supply depots (miles) 750 

15 Other personnel costs ($/person) 0 

*************************** SYSTEM DESIGN PARAMETERS *************************** 

1 Required system confidence level against stockout 95 

2 System assembly cost at given lot size 5204 

3 Lot size for assembly 100 

4 Average LRA weight (lb) 1.5 

5 Fault isolation hardware cost ($/system) 2.35714E+06 

6 Support and test hardware cost ( $/sy stein) 0 

7 Support and test software development cost 0 

8 Average LRA specific software development cost 80000 

9 Technical documentation for system description (pages) 3 

10 Technical documentation for fault isolation (pages) 0 

11 System scheduled maintenance requirement (manhr/wk/ship) 25 

12 Required number of operators per system .....; T 

13 Required no. officers per platform.. 0 

14 Required days C-school for operator training 10 

15 Required days C-school maintenance trng (sys orien + LRA R&R) 10 
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LRA Configuration for SINCGARS 


CHASSIS: 1 

0 Estimated unit cost 1400 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 14006 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

B LRA repair material cost per failure . 20 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 2 


CONTROL: 1 

0 Estimated unit cost . ... 375 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 27855 

3 Mean time to fault isolate, R & R this LRA (hours)... 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


PWR ASSY: 1 

0 Estimated unit cost.. 180 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 25974 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


TUHR/MXR: 1 

0 Estimated unit cost 370 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 12937 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours)...... 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description... 21 

10 Quantity of similar assemblies 1 
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LRA Configuration for SlNCGARS 
IF UEMOL ) : 1 

0 Estimated unit cost 325 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 18868 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


EXCITER: 1 

0 Estimated unit cost 950 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours)... 11737 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 2 


SYNTHSIZER: 1 

0 Estimated unit cost 525 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 15674 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1.5 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 2 


TWO-HIRE: 1 

0 Estimated unit cost...... 160 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 22624 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 0 

8 LRA repair material cost per failure.............. 200 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 3 
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IRA Configuration for SINCGARS 


SWITCH: 1 

0 Estimated unit cost 400 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours). 12837 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies... 2 


REMOTE 1/0: 1 

0 Estimated unit cost 225 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours)... 44444 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


ANT C0NTRL: 1 

0 Estimated unit cost 175 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 28818 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


AUU PWR SUP: 1 

0 Estimated unit cost 210 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 32573 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 

E-9 



LRA Configuration for SINCGARS 
AUD DATA I/O: 1 

0 Estimated unit cost 260 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 14881 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


BATTERY CASE: 1 

0 Estimated unit cost 50 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 86957 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours). 10 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 0 

8 LRA repair material cost per failure 60 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 3 


AUL) DAT C0NTRL: 1 

0 Estimated unit cost 180 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 21978 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 1 

5 LRA duty cycle (decimal ) 1 

6 LRA specific support and test hardware cost 67 

7 Training hours required for repair of LRA 1 

8 LRA repair material cost per failure 30 

9 Pages of documentation for repair description 21 

10 Quantity of similar assemblies 1 


ANTENNA: 1 

0 Estimated unit cost 40 

1 Corresponding lot size.... 100 

2 LRA mean time between failures (hours)... 13158 

3 Mean time to fault isolate, R & R this LRA (hours) 3 

4 Mean time to repair LRA (hours) 2 

5 LRA duty cycle (decimal) 1 

6 LRA specific support and test hardware cost 0 

7 Training hours required for repair of LRA 0 

8 LRA repair material cost per failure 0 

9 Pages of documentation for repair description 0 

10 Quantity of similar assemblies 3 


E-10 



LRA Configuration for S I fiCGARS 


HANDSET: 1 

0 Estimated unit cost 80 

1 Corresponding lot size 100 

2 LRA mean time between failures (hours) 780 


3 Mean time to fault isolate, R S R this LRA (hours) 

4 Mean time to repair LRA (hours) 

5 LRA duty cycle (decimal) 

6 LRA specific support and test hardware cost 

7 Training hours required for repair of LRA 

8 LRA repair material cost per failure..... 

9 Pages of documentation for repair description 

10 Quantity of similar assemblies 


E-ll 


rooj.— iOOOOoo 



CHASSIS 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

1400 lot 

100 MTBF 

14006 

MTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+T E 

67 

Trng hrs 

1 

Repair parts $ 

20 Tech doc pp 

21 QSA 

2 



******************* 

SUPPORT POLICY 

SUMMARIES 

**************** 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

46846.4 

40340.8 

40227.0 

49635.4 

INITIAL COSTS: 

Production 

39356.8 

39356.8 

39356.8 

39356.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

284.5 

280.3 

172.5 

1830.2 

S+TE 

0.0 

80.1 

81.9 

U.O 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

39641.3 

39721.5 

39616.2 

41187.0 

OPERATION AND SUPPORT COSTS: 

Wage 

251.9 

344.9 

335.8 

251.9 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

82.0 

82.0 

82.0 

8196.5 

Repai r 

6871.3 

137.4 

137.4 

0.0 

Transportation 

0.0 

3.1 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Uoc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

7205.1 

619.3 

610.8 

8448.4 


Support Posture Assigned: Local Repair 

♦ ★★★♦★★♦★★★★★★★★★★★★★★♦♦★★★★★★★★★★★it** 


E-12 



CONTROL 


INPUT DATA 

********** 

Quantity 

1 

UC(lot) 

375 lot 

100 MTBF 

27855 

NT HR 

3.0 

MTTK 

1.0 Outy cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



★★★★★★★ ★★★★★★ ★*★★★★ 
Contractor 

SUPPORT POLICY 
Mil itary 

SUMMARIES 

Local 

**************** 


Depot 

Depot 

Repai r 

Discard 

LIFE CYCLE COST 

14184.4 

11016.6 

11001.7 

12029.9 

INITIAL COSTS: 

Production 

10542.0 

10542.0 

10542.0 

10542.0 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

49.7 

48.6 

33.6 

257.4 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

'♦TOTAL 

10591.7 

10675.0 

10662.6 

10799.4 


OPERATION AND SUPPORT COSTS: 
Wage 

126.7 

173.4 

168.9 

126.7 

Training 

0.0 

U.l 

2.7 

0.0 

Spares 

11.0 

11.0 

11.0 

1103.9 

Repair 

3455.0 

103.6 

103.6 

0.0 

Transportation 

0.0 

1.6 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

3592.7 

341.6 

339.1 

1230.6 

Support Posture Assigned: Military Depot 
**************************************** 





E-13 



IF UEMOD 


INPUT DATA 
********** 

Quantity 
ilTRR 
Trng hrs 


1 UC(lot) 325 

3.0 MTTR 1.0 

1 Repair parts $ 30 


lot 

100 MTBF 

Duty cycle 

1.0 S+TE 

Tech doc pp 

21 QSA 


18868 

67 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

★★★★★★★★★★★★★★★★ 


Contractor 

Military 

Local 



Depot 

Depot 

Repair 

Discard 

LIFE CYCLE COST 

14493.2 

9752.3 

9725.4 

11057.0 

INITIAL COSTS: 





Production 

9136.4 

9136.4 

9136.4 

9136.4 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

55.0 

54.0 

30.0 

321.1 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

9191.4 

9274.8 

9253.4 

9457.5 

OPERATION AND SUPPORT COSTS: 




Wage 

187.0 

256.0 

249.3 

187.0 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

14.1 

14.1 

14.1 

1412.4 

Repair 

5100.6 

153.0 

153.0 

0.0 

Transportation 

0.0 

2.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

5301.7 

477.4 

472.0 

1599.4 

Support Posture Assigned: 

Military Depot 





*************** ************************* 



TUNR/MXR 


INPUT DATA 


Quantity 

1 

UC(lot) 

370 lot 

100 MTBF 

12937 

MTKK 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

**************** 


Contractor 

Depot 

Mil itary 
Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

18213.3 

11236.7 

11200.1 

13538.3 

INITIAL COSTS: 
Production 

10401.4 

10401.4 

10401.4 

10401.4 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

76.6 

75.5 

45.9 

519. U 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

O.C 

♦♦TOTAL 

10478.1 

10561.4 

10534.3 

10920.4 

OPERATION AND SUPPORT COSTS: 

Wage 

272.7 

373.4 

363.6 

272.7 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

Repair 

23.5 

23.5 

23.5 

2345. 

7439.0 

223.2 

223.2 

o.. 

Transportation 

0.0 

3.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

7735.2 

675.3 

665.8 

2617.9 


Support Posture Assigned: Military Depot 
**************************************** 


E-15 



PUR ASSY 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

180 lot 

100 MTBF 

25974 

HTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

************'***'* 


Contractor 

Military 

Local 



Oepot 

Oepot 

Repair 

Discard 

LIFE CYCLE COST 

8930.8 

5524.9 

5516.8 

5895.6 

INITIAL COSTS: 





Production 

5060.2 

5060.2 

5060.2 

5060.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

23.9 

23.9 

16.2 

131.4 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

5084.1 

5168.5 

5163.3 

5191.6 

OPERATION AND SUPPORT COSTS: 




Wage 

135.8 

186.0 

181.1 

135.8 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.7 

5.7 

5.7 

568.3 

Repair 

3705.2 

111.2 

111.2 

0.0 

Transportation 

0.0 

1.7 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

3846.7 

356.4 

353.5 

704.1 

Support Posture Assigned: 

Military Depot 





**************************************** 


E-14 



EXCITER 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

950 1 ot 

100 MTBF 

11737 

MTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

2 



******************* 

SUPPORT POLICY 

SUMMARIES 

**************** 


Contractor 

Depot 

Military 

Depot 

Local 
Repai r 

Di scard 

LIFE CYCLE COST 

35476.6 

27774.0 

27681.1 

35102.8 

INITIAL COSTS: 
Production 

26706.4 

26706.4 

26706.4 

26706.4 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

203.6 

203.6 

119.1 

1458.7 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

26910.0 

26994.4 

26912.4 

28165.1 

OPERATION AND SUPPORT COSTS: 

Wage 

300.6 

411.6 

400.8 

300.6 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

66.4 

66.4 

66.4 

6637.1 

Repair 

8199.6 

246.0 

246.0 

0.0 

Transportation 

0.0 

3.7 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

8566.6 

779.6 

768.7 

6937.7 


Support Posture Assigned: Local Repair 
************************************** 


E-17 



SYNTHSIZER 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

525 lot 

100 MTBF 

15674 

HTRR 

3.0 

MTTR 

1.5 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

2 



******************* 

Contractor 

SUPPORT POLICY 
Mi li tary 

SUMMARIES 

Local 

★ ★★ ★★ ★ ★★ ★ * ★ ★ ★ ★ ★ 


Depot 

Depot 

Repair 

Discard 

LIFE CYCLE COST 

21245.9 

15552.3 

15500.0 

18342.8 

INITIAL COSTS: 

Product i on 

14758.8 

14758.8 

14758.8 

14758.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

94.5 

92.9 

49.3 

612.3 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

14853.3 

14936.1 

14895.1 

15371.1 


OPERATION AND SUPPORT COSTS: 


Wage 

225.1 

349.8 

337.6 

225.1 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

27.5 

27.5 

27.5 

2746.6 

Repair 

6140.0 

184.2 

184.2 

0.0 

Transportation 

0.0 

2.8 

0.0 

O.C 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

6392.6 

616.1 

604.9 

2971.7 


Support Posture Assigned: Local Repair 
************************************** 


E-18 



TWO-WIRE 


I NPUT DATA 
★ ★ ★★ ★* ★ ★ ★ 


Quantity 

1 

UC(lot) 

160 

lot 

100 MTBF 

22624 

MTKK 

3.0 

MTTR 

1.0 

Duty cycle 

1.0 S+TE 

67 

Trng hrs 

0 

Repair parts $ 

200 

Tech doc pp 

21 QSA 

3 



it irk it kirk irk irk irk it irk it irk 

Contractor 

SUPPORT POLICY 
Military 

SUMMARIES 

Local 

★ * * ★ * ★ * * ★ * * * ★ ★ * ★ 


Depot 

Depot 

Repair 

Discard 

LIFE CYCLE COST 

8936.4 

5728.9 

5715.9 

5366.3 

INITIAL COSTS: 

Production 

4497.9 

4497.9 

4497.9 

4497.9 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

22.9 

22.9 

14.5 

132.5 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

**T0TAL 

4520.8 

4605.1 

4598.5 

4630.4 

OPERATION AND SUPPORT COSTS: 

Wage 

155.9 

213.5 

207.9 

155.9 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

5.8 

5.8 

5.8 

579.9 

Repair 

4253.8 

850.8 

850.8 

0.0 

Transportation 

0.0 

1.9 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

4415.6 

1123.8 

1117.4 

735.9 


Support Posture Assigned: Discard 
********************************* 


E-19 



SWITCH 


INPUT DATA 
********** 


Quantity 

1 

IJC(lot) 

400 lot 

100 MTBF 

12837 

HTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

2 


******************* 

SUPPORT POLICY 

SUMMARIES 

‘kkkkkickk'kkkkkk-kk 


Contractor 

Mi 1 i ta ry 

Local 



Depot 

Depot 

Repai r 

Discard 

LIKE CYCLE COST 

19125.0 

12093.0 

12053.9 

14639.6 

INITIAL COSTS: 

Production 

11244.8 

11244.8 

11244.8 

11244.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

82.9 

81.7 

49.7 

564.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Uoc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

11327.7 

11410.9 

11381.4 

11809.6 

OPERATION AND SUPPORT COSTS: 

Wage 

274.8 

376.3 

366.4 

274.8 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 
Repai r 

25.6 

25.6 

25.6 

2555.1 

7497.0 

224.9 

224.9 

0.0 

Transportation 

0.0 

3.4 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

Support Posture Assigned: Local 

7797.4 

Repair 

682.1 

672.5 

2829.9 


************************************** 


E-20 



REMOTE I/O 


INPUT DATA 

irkirk'k'kiritirit 


Quantity 

1 

UC(lot) 

225 lot 

100 MTBF 

44444 

MTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



Contractor 

Depot 

SUPPORT POLICY 
Military 
Depot 

SUMMARIES 

Local 

Repair 

'k'k'k'kirkirkiriticirkltirit 

Discard 

LIKE CYCLE COST 

8596.5 

6662.0 

6656.1 

6919.9 

INITIAL COSTS: 

Production 

6325.2 

6325.2 

6325.2 

6325.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

22.4 

21.7 

13.4 

100.2 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

6347.6 

6431.3 

6425.5 

6425.4 


OPERATION AND SUPPORT COSTS: 
Wage 

79.4 

108.7 

105.8 

79.4 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

4.2 

4.2 

4.2 

415.1 

Repair 

2165.4 

65.0 

65.0 

0.0 

Transportation 

0.0 

1.0 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

2248.9 

230.7 

230.5 

494.5 


Support Posture Assigned: Military Uepot 
**************************************** 


E-21 



ant corim 


INPUT DATA 

★★★★★★★★★Hr 


Quantity 

1 

UC(lot) 

175 lot 

100 MTBF 

28818 

MTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

******** -ft*****-*"* 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

8409.7 

5352.9 

5346.2 

5656.8 

INITIAL COSTS: 

Production 

4919.6 

4919.6 

4919.6 

4919.6 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

23.2 

22.7 

15.7 

116.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

4942.8 

5026.6 

5022.2 

5036.4 


OPERATION AND SUPPORT COSTS: 


Wage • 

122.4 

167.6 

163.2 

122.4 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.0 

5.0 

5.0 

498.0 

Repair 

3339.5 

100.2 

100.2 

0.0 

Transportation 

0.0 

1.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦TOTAL 

3466.9 

326.2 

324.0 

620.4 


Support Posture Assigned: Military Depot 
**************************************** 


E-22 



AUD PUR SUP 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

210 lot 

100 MTBF 

32573 

f-lTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 


******************* SUPPORT POLICY SUMMARIES **************** 



Contractor 

Depot 

Military 

Depot 

Local 
Repai r 

Discard 

LIFE CYCLE COST 

8998.7 

6309.8 

6297.2 

6666.9 

INITIAL COSTS: 

Production 

5903.5 

5903.5 

5903.5 

5903.5 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

27.0 

26.4 

12.8 

126.4 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

**T0TAL 

5930.6 

6014.3 

6003.2 

6029.9 

OPERATION AND SUPPORT COSTS: 

Wage 

108.3 

148.3 

144.4 

108.3 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.3 

5.3 

5.3 

528.7 

Repair 

2954.6 

88.6 

88.6 

0.0 

Transportation 

0.0 

1.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

**T0TAL 

3068.1 

295.5 

293.9 

637.0 


Support Posture Assigned: Military Depot 
**************************************** 


E-23 



AUD DATA I/O 


input data 
********** 


Quantity 

1 

UC(lot) 

260 lot 

100 MTBF 

14881 

MTRR 

3.0 

MTTR 

1.0 Uuty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

iclcirkirirkiririrkirititirk 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

14080.4 

8033.2 

8000.7 

9301.7 

INITIAL COSTS: 

Production 

7309.1 

7309.1 

7309.1 

7309.1 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

52.7 

51.9 

24.6 

322.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

**TOTAL 

7361.8 

7445.4 

7420.6 

7631.9 


OPERATION AND SUPPORT COSTS: 


Wage 

237.1 

324.6 

316.1 

237.1 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

14.3 

14.3 

14.3 

1432.7 

Repair 

6467.2 

194.0 

194.0 

0.0 

Transportation 

0.0 

2.9 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

6718.6 

587.8 

580.0 

1669.8 


Support Posture Assigned: Military Depot 
**************************************** 


E-24 



BATTERY CASE 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

50 

lot 

100 MTBF 

86957 

MTKR 

3.0 

MTTR 

10.0 

Duty cycle 

1.0 S+TE 

67 

Trng hrs 

0 

Repair parts $ 

60 

Tech doc pp 

21 QSA 

3 



******************* 

SUPPORT POLICY 

SUMMARIES 

★★ ★ ★★ ★ ★ ★ ★★★ ★* * 


Contractor 

Oepot 

Military 

Depot 

Local 
Repai r 

Discard 

LIFE CYCLE COST 

2556.6 

1802.7 

1790.1 

1505.4 

INITIAL COSTS: 

Production 

1405.6 

1405.6 

1405.6 

1405.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

3.2 

3.2 

2.9 

12.1 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

♦♦TOTAL 

1408.8 

1493.1 

1494.6 

1417.7 


OPERATION AND SUPPORT COSTS: 


Wage 

40.6 

190.4 

175.8 

40.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

0.5 

0.5 

0.5 

47.1 

Repair 

1106.7 

66.4 

66.4 

0.0 

Transportation 

0.0 

0.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

♦♦TOTAL 

1147.8 

309.6 

295.6 

87.7 


Support Posture Assigned: Oiscard 
************** ************* ****** 


E-25 



AUD DAT CONTRL 


INPUT UATA • 
********** 

Quantity 

1 

UC(lot) 

180 lot 

100 MTBF 

21978 

MTRR 

3.0 

MTTR 

1.0 Duty cycle 

1.0 S+TE 

67 

Trng hrs 

1 

Repair parts $ 

30 Tech doc pp 

21 QSA 

1 



******************* 

SUPPORT POLICY 

SUMMARIES 

★★ ★ ★ * ★★★ ★ ★ ★★ ★ ★★ ★ 


Contractor 

Depot 

Military 

Depot 

Local 
Repai r 

Discard 

LIFE CYCLE COST 

9635.4 

5585.5 

5571.2 

6048.4 

INITIAL COSTS: 

Production 

5060.2 

5060.2 

5060.2 

5060.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

29.2 

29.2 

16.4 

156.2 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech. Doc. 

0.0 

4.2 

4.2 

0.0 

**T0TAL 

5089.3 

5173.7 

5163.5 

5216.3 

OPERATION AND SUPPORT COSTS: 

Wage 

160.5 

219.8 

214.0 

160.5 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

6.7 

6.7 

6.7 

671.6 

Repair 

4378.9 

131.4 

131.4 

0.0 

Transportation 

0.0 

2.0 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

o.c 

Tech. Doc. 

0.0 

2.6 

2.6 

0.0 

**T0TAL 

4546.1 

411.8 

407.7 

832.1 


Support Posture Assigned: Military Depot 
**************************************** 


E-26 



ANTENNA 


INPUT DATA 
********** 

Quantity 

1 

UC(lot) 

40 

lot 

100 

MTBF 

13158 

MTRR 

3.0 

HTTR 

2.0 

Duty cycle 

1.0 

S+TE 

0 

Trng hrs 

0 

Repair parts $ 

0 

Tech doc pp 

0 

QSA 

3 



******************* 

SUPPORT POLICY 

SUMMARIES 

* ★ * * * * * * * * * * * * * ★ 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

8717.5 

1733.7 

1707.9 

1697.2 

INITIAL COSTS: 

Production 

1124.5 

1124.5 

1124.5 

1124.5 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

8.3 

8.2 

5.0 

55.3 

S+TE 

0.0 

80.0 

80.0 

0.0 

Tech. Doc. 

0.0 

0.0 

0.0 

0.0 

**T0TAL 

1132.8 

1212.6 

1209.4 

1179.8 

OPERATION AND SUPPORT COSTS: 

Wage 

268.1 

466.1 

446.9 

268.1 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

2.5 

2.5 

2.5 

249.3 

Repair 

7314.1 

0.0 

0.0 

0.0 

Transportation 

0.0 

3.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

49.2 

0.0 

Tech. Doc. 

0.0 

0.0 

o.n 

0.0 

**T0TAL 

7584.7 

521.1 

498.5 

517.4 


Support Posture Assigned: Discard 

*******************t************* 


E-27 



HANDSET 


INPUT DATA 
★★★★★★★**★ 

Quantity 1 UC(lot) 80 lot 

NTKK 3.0 MTTK 2.0 Duty cycle 

Trng hrs 0 Repair parts $ 0 Tech doc pp 


100 MTBF 
1.0 S+TE 
0 QSA 


780 

0 

3 


******************* SUPPORT POLICY SUMMARIES **************** 



Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE CYCLE COST 

130370.0 

10511.9 

10063.6 

16903.1 

INITIAL COSTS: 

Production 

2249.0 

2249.0 

2249.0 

2249.0 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

130.6 

130.6 

63.1 

1721.0 

S+TE 

0.0 

80.0 

£0.0 

0.0 

Tech. Doc. 

0.0 

0.0 

0.0 

0.0 

♦♦TOTAL 

2379.6 

2459.6 

2392.1 

3969.9 

OPERATION AND SUPPORT COSTS: 

Wage 

4522.9 

7863.5 

7538.2 

4522.9 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

84.1 

84.1 

84.1 

8410.3 

Repair 

123383.0 

0.0 

0.0 

0.0 

Transportation 

0.0 

55.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

49.2 

0.0 

Tech. Doc. 

0.0 

0.0 

0.0 

0.0 

♦♦TOTAL 

127990.0 

8052.3 

7671.5 

12933.2 


Support Posture Assigned: Discard 
********************************* 


E-28 



BATTERY 


INPUT DATA 
********** 


Quantity 

1 

UC(lot) 

80 

lot 

100 MTBF 

24 

flTRR 

0.3 

MTTR 

2.0 

Duty cycle 

1.0 S+TE 

0 

Trng hrs 

0 

Repair parts $ 

0 

Tech doc pp 

0 QSA 

3 



* ********* *** *** ** * 

SUPPORT POLICY 

SUMMARIES 

**************** 


Contractor 

Military 

Local 



Depot 

Depot 

Repair 

Discard 

LIFE CYCLE COST 

%4030330.0 

130887.0 

116801.0 

342672.0 

INITIAL COSTS: 

Production 

2249.0 

2249.0 

2249.0 

2249.0 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

3153.5 

3153.5 

. 1442.7 

54839.4 

S+TE 

U.O 

80.0 

80.0 

0.0 

Tech. Uoc. 

0.0 

0.0 

0.0 

0.0 

♦♦TOTAL 

5402.5 

5482.5 

3771.7 

57088.4 

OPERATION AMD SUPPORT COSTS: 

Wage 

12249.6 

120818.0 

110247.0 

12249.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

2733.3 

2733.3 

2733.3 

273334.0 

Repair 

24009950.0 

0.0 

0.0 

0.0 

Transportation 

0.0 

1804.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

49.2 

0.0 

Tech. Doc. 

0.0 

0.0 

0.0 

0.0 

♦♦TOTAL 

24024930.0 

125405.0 

113029.0 

285584.0 


Support Posture Assigned: Discard 
********************************* 


E-29 



